Pure Master Slave Simplified Topology

I am not actually going to go into Broker topologies, there are many great resources for that such as this by Bruce Snyder: http://www.slideshare.net/bruce.snyder/messaging-with-activemq-presentation or http://activemq.apache.org/topologies.html, all great stuff. This example uses a store and forward topology, or, distributed queues, and incorporates basic authentication: My use case was to handle down JMS Servers. What I needed to do was implement failover as well as master slave strategies and a topology for message redundancy in case of hardware failure, etc. The client could not have any message loss. With failover, you can see how ActiveMQ switches from the main broker to the second, third, etc on failure. I have a case of four JMS servers in production, each server it is on is load balanced. There are just a few configurations to add or modify in order to set up JMS Failover with Master/Slave for your broker topology. Here is a basic configuration. For this use case, all JMS servers are configured as standalone versus embedded.

I. Client URI

You will need to add the Failover protocol, either with a basic URI pattern or a composite. In this use case, there are load balanced servers in Production and multiple Development and QA environments which require different configurations for master/slave and failover.

In your application’s properties file for messaging add a modified version of this with your mappings:
activemq.broker.uri=failover://(tcp://localhost:61616,tcp://slaveh2:61616,tcp://master2:61616,tcp://slave2:61616,network:static://(tcp://localhost:61616,tcp://master2:61616,tcp://slave2:61616))?randomize=false

Note: I set connections as locked down (static) communication configurations vs multicast or dynamic discovery so that I know exactly what servers can communicate with each other and how. Also this is assuming you have one set per environment to account for mapping the appropriate IP’s in development, qa, production, dr, etc.

Note: Do not configure networkConnections for master slave, they are handled on the slave with the following configuration:
<masterConnector remoteURI= "tcp://masterhost:61616" userName="wooty" password="woo"/>

II. Spring Configuration

<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory"
destroy-method="stop">
<property name="connectionFactory">
<bean class="org.apache.activemq.spring.ActiveMQConnectionFactory">
<constructor-arg value="${amq.broker.uri}"/>
<property name="userName" value="${activemq.username}"/>
<property name="password" value="${activemq.password}"/>
</bean>
</property
</bean>

III. Broker Configuration

Master

<broker brokerName="{hostname}" waitForSlave="true" xmlns="http://activemq.apache.org/schema/core" dataDirectory="${activemq.base}/data">
<networkConnectors>
<!-- passed in by the client broker URI so you can easily manager per environment: sweet -->
</networkConnectors>
<transportConnectors>
<!-- TCP uses the OpenWire marshaling protocol to convert messages to stream of bytes (and back) -->
<transportConnector name="tcp" uri="tcp://localhost:61616?trace=true" />
<transportConnector name="nio" uri="nio://localhost:61618?trace=true" />
<!-- <transportConnector name="ssl" uri="ssl://localhost:61617"/>
<transportConnector name="http" uri="http://localhost:61613"/
<transportConnector name="https" uri="https://localhost:61222"/> -->
<transportConnectors>
</transportConnectors>

<!-- Basic security and credentials -->
<plugins>
<simpleAuthenticationPlugin>
<users>
<authenticationUser username="system" password="manager" groups="admin, publishers,consumers"/>
</users>
</simpleAuthenticationPlugin>
</plugins>
..more configuration
</broker>

Slave: for ActiveMQ 4.1 or later which also allows for authentication as show below

<broker brokerName="{hostname}Slave" deleteAllMessagesOnStartup="true" xmlns="http://activemq.apache.org/schema/core">
<transportConnectors>
<transportConnector uri="tcp://localhost:61616"/>
</transportConnectors>
<services>
<masterConnector remoteURI= "tcp://masterhost:62001" userName="wooty" password="woo"/
</services>
<!-- Basic security and credentials -->
<plugins>
<simpleAuthenticationPlugin>
<users>
<authenticationUser username="system" password="manager" groups="admin, publishers,consumers"/>
</users>
</simpleAuthenticationPlugin>
</plugins>
</broker>

I wanted to expose mbeans related to application monitoring and services such as a JMS broker, users in session, etc and remote that data to the client which in my case was Flex.  First I checked out using annotations for automated registration and mbean creation but that wouldn’t help me export to a remoting format..without creating my own classes. Something for another post, but what I did was create a package with some classes, which I will explain here. You can certainly deliver this strategy more minimally but I’ve found that to come back and bite me later so I’ve added in some type safety to the proxy creation of the MBeans.

The Basic Components There are a few important components you need to set this up as a framework 1. An MBeanConnection 2. A type-safe proxy generator 3. An applicable JMX service URL, Port, Hostname To export mbeans to the client 1. ObjectName that can be connected to if they are services 2. Ports for those services where applicable (i.e. a broker) 3. Interfaces to proxy I. The Goal: export mbean data by remoting to the client I created a service which is mirrored as a Flex actionscript class for remoting which encapsulates the calls for various system monitoring data via jmx: This I registered for remoting in the framework of choice, in my case it was Flex. I have a custom annotation that handles this which I don’t include in this snippet. public class ClientMBeanService implements IClientMBeanService { ..view the code… } II. The Delegate With the idea of each component to be exported create a Delegate class that implements the interface dictating the contract that it must implement a method to provide the port, the ObjectName and export the mbean: public class BrokerDelegate implements IMBeanDelegate { ..view the code… } All Delegates must implement public interface IMBeanDelegate { ..view the code… } III. For Remote and Local MBeanConnections public class MBeanConnection { ..view the code… } IV. The Cool Part: Get a Typesafe Proxy Intance of The MBean public final class MBeanProxyInstance { ..view the code… } V. Typesafing: I did not originally write this class, only modified it for Java 6, but the crux is: public class TypeSafeMBeanProxy { ..view the code… }

If you just want to easily export mbeans to jconsole or an external profiling tool this is pretty sweet:

I. Create a Spring schema config file for JMX

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

<!-- This bean needs to be eagerly pre-instantiated in order for the exporting to occur: -->
<!-- ***************** Autodetects Annotated Beans JMX Exposure *********************** -->
<bean id="annotationalMbeanExporter" class="org.springframework.jmx.export.MBeanExporter" lazy-init="false">
<property name="autodetect" value="true"/>
<property name="namingStrategy" ref="namingStrategy"/>
<property name="assembler" ref="assembler"/>
<!-- one of many optional config options -->
<property name="registrationBehaviorName" value="REGISTRATION_IGNORE_EXISTING"/>
</bean>
<bean id="attributeSource" class="org.springframework.jmx.export.annotation.AnnotationJmxAttributeSource"/>
<bean id="assembler" class="org.springframework.jmx.export.assembler.MetadataMBeanInfoAssembler">
<property name="attributeSource" ref="attributeSource"/>
</bean>
<bean id="namingStrategy" class="org.springframework.jmx.export.naming.MetadataNamingStrategy">
<property name="attributeSource" ref="attributeSource"/>
</bean>
</beans>

II. Annotate a component, bean, etc

@Service
@ManagedResource(objectName = "bean:name=applicationMonitor")
public class ClientMBeanService implements IClientMBeanService {

@ManagedOperation(description = "Monitor the JMS broker")
public BrokerStatistics exposeBroker() {
return new BrokerDelegate().exposeMBean();
}
}

@Component
@ManagedResource(objectName = "spring:name=broker", description = "ActimemMQ Broker")
public class BrokerStatistics {

private String brokerId;
private String brokerName;
private long totalMessages;
private long totalConsumers;
private long totalTopics;
private long totalQueues;
private List<Map> topics;
private List<Map> queues;

@ManagedAttribute
public String getBrokerId() {
return brokerId;
}
...etc
}

The annotational configuration in the mbean exporter allows automatic registration and mbean creation of beans. It’s that simple.

I needed to set up and test ActiveMQ failover URI’s and play around with Broker topologies for optimization in a safe environment (in my control environment where no other app servers or brokers other than this env could access). I have a Mac laptop with multiple VM operating systems so that I can easily work with any client/company requirements for a current project (ubuntu, windoz, etc). Here’s what I did to set up 2 local test Brokers:

1. Know both native and virtual operating system host names. IP addresses will not work

Explicitly setting IP addresses in the tc config will fail on the native OS as it attempts to bind to the virtual IP

2. Give each Broker a unique name

Standard procedure for any multiple Broker topology on a network

Installing, configuring, and running ActiveMQ is a no brainer so I won’t cover that here but the specific, simple configuration allowing me to run multiple instances on one machine were:

3. Super basic configuration of the connectors: In the activemq.xml Spring config file

Note: what I did here is set up two transport protocols, one for TCP, one for NIO

Instance One: on the native laptop OS

<networkConnectors>
<networkConnector name=”nc-1″ uri=”multicast://default” userName=”system” password=”manager”/>
</networkConnectors>

<transportConnectors>
<transportConnector name=”tcp” uri=”tcp://nativeHostName:61616?trace=true” discoveryUri=”multicast://default”/>
<transportConnector name=”nio” uri=”nio://nativeHostName:61618?trace=true” discoveryUri=”multicast://default”/>
</transportConnectors>

Instance Two: on the virtual machine’s OS

<networkConnectors>
<networkConnector name=”nc-2″ uri=”multicast://default” userName=”system” password=”manager”/>
</networkConnectors>

<transportConnectors>
<transportConnector name=”tcp” uri=”tcp://virtualHostName:61616?trace=true” discoveryUri=”multicast://default”/>
<transportConnector name=”nio” uri=”nio://virtualHostName:61618?trace=true” discoveryUri=”multicast://default”/>
</transportConnectors>

It’s that easy.

4. Start the native then virtual ActiveMQ servers

5. Open a browser on both OS’s and if you enabled the web console, at http://localhost:8161/admin/ you will see the broker interface with each unique broker name.

I’ve just started staging my new project, SuMQ, and appreciate patience as this will take me a while to get the code standardized and on google.

SuMQ is a light-weight enterprise messaging framework built in Java, leveraging Spring, JMS, and ActiveMQ. It plugs into Flex Messaging via BlazeDS for the client. This can also be configured for other clients aside from Flex.

The sample will be ready for clustered BlazeDS instances and load balanced Application Servers.

code.google.com/p/sumq