Helena Edelson

Enterprise & Cloud Messaging | Spring | Enterprise Integration | RabbitMQ | Esper | CEP | Queueing Systems, Channels & Flows | JMS | Security | AOP | Flex

  • Cloud

  • Topics

  • Archives



RabbitMQ and Erlang Intallation for Mac OSX

Posted by Helena Edelson on June 11th, 2010

I’ve been a linux developer for years so I was all thumbs having to look up everything when setting up a new dev environment on a mac. Here’s some tips for setting up a local RabbitMQ server to play with (i.e. not using macports locally, not the EC2 install etc):

Install and build Erlang

First, Check for the latest Erlang GA Release and note the version.

  1. cd to the dir you wish to install into
  2. wget http://erlang.org/download/otp_src_{version}.tar.gz
  3. tar xzvf otp_src_{version}.tar.gz
  4. cd otp_src_{version}
  5. NOTE: run ./configure –help to see what opts you want to run with
  6. ./configure
  7. make
  8. sudo make install
  9. cd ../

Install RabbitMQ Server

First check the latest version of RabbitMQ Server and note the filename.

  1. wget http://www.rabbitmq.com/releases/rabbitmq-server/v1.7.2/rabbitmq-server-generic-unix-1.7.2.tar.gz
  2. tar xzvf rabbitmq-server-generic-unix-1.7.2.tar.gz
  3. Complete the install and config from http://www.rabbitmq.com/install.html#generic-unix, making sure the logs, clustering, data and config dirs are set up
  4. Start RabbitMQ: run rabbit../sbin/rabbitmq-server

If you need a few things first

  1. First, if you don’t have it, download Apple XCode so you can compile Erlang with gcc. FYI the XCode download could not take longer, start it and go build an enterprise app in ROO, deploy it to the cloud, and when you’re done it may have completed. If I knew more about it I’d just download the particular compiler needed which is some form of C compiler. If I find out I’ll update this for a simpler install. if you know, send me a tweet: @hollyedelson
  2. While that’s downloading, if you don’t have it, get the latest version of wget from http://ftp.gnu.org/pub/gnu/wget
  3. On the command line:
    1. tar -xzf wget-{version}.tar.gz
    2. cd wget{version}
    3. ./configure
    4. make
    5. sudo make install
  4. Now run the steps from Version A above.

Find Out More

rabbitmq.com erlang.org

Share/Save/Bookmark

Posted in Cloud, Erlang, Messaging, RabbitMQ, Spring ROO | 1 Comment »

JMX and MBean Support With Spring

Posted by Helena Edelson on May 17th, 2010

The context of this post is simply about how to use the Spring Framework to export your Spring-managed pojos for management and monitoring via JMX. Later I’ll post on using Hyperic, in the cloud, etc. First things first – as this is an update from a similar post of mine from ‘08 or early ‘09.

Let’s start by adding the spring jmx dependency, which will be something like this, depending on the repos you are using:

<dependency>
<groupId>org.springframework</groupId>
<artifactId>org.springframework.core</artifactId>
<version>${spring.framework.version}</version>
</dependency>

Service Pojo

Now let’s set up some java classes for management. I have a business service that I want to monitor and a pojo to instrument.

@BusinessService
@ManagedResource(objectName = "bean:name=inventoryManager", description = "Inventory Service",
        log = true, logFile = "oms.log", currencyTimeLimit = 15, persistPolicy = "OnUpdate", persistPeriod = 200,
        persistLocation = "foo", persistName = "bar")
public class InventoryServiceImpl implements InventoryService {

    @Autowired private InventoryDao inventoryDao;

    @ManagedOperation(description = "Add two numbers")
    @ManagedOperationParameters({
            @ManagedOperationParameter(name = "x", description = "The first number"),
            @ManagedOperationParameter(name = "y", description = "The second number")})
    public int add(int x, int y) {
        return x + y;
    }

    @ManagedOperation(description = "Get inventory levels")
    @ManagedOperationParameters({@ManagedOperationParameter(name = "product", description = "The Product)})
    public long getInventoryLevel(Product product) {
        return getInventoryLevel(product.getSkew());
    }
}

First, let’s peak into my @BusinessService annotation in case you are wondering:

@Transactional
@Service
@Target({ElementType.TYPE, ElementType.ANNOTATION_TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Inherited
@Documented
public @interface BusinessService {
}

Annotating any of my service layer pojo’s makes them both transactional and with an instance created in the Spring context.

Entity Pojo

Now here is a simple pojo as an entity or javabean, what have you:

@ManagedResource(objectName = "bean:name=myPojoEntity", description = "My Managed Bean", log = true,
        logFile = "oms.log", currencyTimeLimit = 15, persistPolicy = "OnUpdate", persistPeriod = 200,
        persistLocation = "foo", persistName = "bar")
public class MyPojo {

    private long somethingToTuneInRuntime;

    /* Creates a writeable attribute for managing */
    @ManagedAttribute(description = "Tunable In Runtime Attribute",
            currencyTimeLimit = 20,
            defaultValue = "bar",
            persistPolicy = "OnUpdate")
    public void setSomethingToTuneInRuntime(long value) {
        this.somethingToTuneInRuntime = value;
    }

    @ManagedAttribute(defaultValue = "foo", persistPeriod = 300)
    public String getSomethingToTuneInRuntime() {
        return somethingToTuneInRuntime;
    }
}

Spring Config

Now let’s configure Spring to autoregister our pojos to export and manage/monitor:
Create a jmx-context.xml file in your WEB-INF/* dir

Add: <context:mbean-export/>

Activates default exporting of MBeans by detecting standard MBeans in the Spring
context as well as @ManagedResource annotations on Spring-defined beans.
Rather than defining an MBeanExporter bean, just provide this single element. I could walk you through a simple, simpler and simplest config of spring jmx but with annotational config in the simplest requirements, this is all you need to do to get up and running. If you need object naming in multiple vm situations, you can easily do that and other things too but, that’s out of scope for this post ;)

Share/Save/Bookmark

Posted in Spring, Spring JMX | No Comments »

Future Posts: Esper, RabbitMQ for Cloud Messaging

Posted by Helena Edelson on May 9th, 2010

If I had more time I would add a few new posts on esper and rabbitmq for messaging in the cloud but I just got home from 2 weeks of coast-to-coast consulting and technology presentations. Pretty exhausting. I hope to have time to post on these and a few other fun technologies. I’ve got some code samples and tests I might post with that, plus some cools stuff on esper queries with Spring Integration.

Share/Save/Bookmark

Posted in Concurrency, Erlang, Java, Messaging, RabbitMQ | No Comments »

Spring Sample App

Posted by Helena Edelson on April 14th, 2010

I’ve been traveling a lot as a Senior Consultant with SpringSource and many of the engineers that have been at my private and public engagements on core spring and enterprise integration with spring have requested a copy of my sample app that I use to demonstrate configurations, run demos and tests, etc.

This week I’m finally getting it finished and available to check out from svn. I’ve had a project set up in one of our public-facing repositories, the code will be available next week. Shoot me an email via my SpringSource address – attendees have that email, or find me on linkedin.com

May 10 I officially move into engineering!

Share/Save/Bookmark

Posted in Java | No Comments »

Spring DAO Exception Translation with @Repository

Posted by Helena Edelson on April 5th, 2010

In Spring, when you mark a DAO/Repository classes with the Spring stereotype annotation, @Repository:

@Repository
public class MyDaoImpl implements MyDao { .. }

spring creates an instance of it in the IoC container as with any other stereotype annotation (@Component, @Service) but it can also add exception translation if you explicitly add this bean declaration to your config:
<bean class=”org.springframework.dao.annotation.PersistenceExceptionTranslationPostProcessor”/>

This enables translation of native runtime resource exceptions that would otherwise be vendor-specific (database (oracle, etc), orm (hibernate, jpa, etc) to Spring’s runtime exception hierarchy regardless of what vendors you use over time. As it is simply enabling a feature in the Spring Framework vs something you would use, it does not require a bean id.

Also as the stereotype annotations are meta annotations you can do this to enrich behavior easily

@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Repository
@Primary
@Lazy
public @interface MyRepository {
}
@MyRepository
public class OtherFooDao implements FooDao { .. }

Share/Save/Bookmark

Posted in Annotations, Spring | No Comments »

Spring Framework 3.0.2 is Released

Posted by Helena Edelson on April 4th, 2010

Read Juergen’s blog post about the latest release and get the code

Share/Save/Bookmark

Posted in Spring | No Comments »

Patterns of Scalability and Pathways in Systems

Posted by Helena Edelson on April 4th, 2010

Note: this is just a draft – I was trying to fall asleep, started to think, and this is what I thought about.

In college I wrote a 300-page senior thesis entitled, “Energy Pathways In Biological Systems”. It was within the context of genetics to microbiology up through complex pathways of micro-climates to ecosystems and even pathways of migratory animals (Arctic Wolves that cover at least 1,000 mile territories to Terns and Whales that have annual migration patterns covering half the earth). For each there is movement of elements in space and time. I had a blast researching and writing it but my fascination revolved around that shared concept over seemingly vast discrepancies of scale that were actually sharing massive similarities, being only sizable to other scales, can only be in a relative framework of complexity.

Think of systems as an atom. There are layers or levels and activity going on all the time. Now think of this atomic model with pathways repetitively used for resources to move, kind of like corridors. Resources behave differently from other resources, thus the corridors are different, the speed of motion is different and the size is different. Also production and consumption of those resources is different. Entropy works differently based on environment, among other factors.

The pathways of genetic information through a cell move in a seemingly small scale to Nitrogen pathways in a rain forest but the complexity in a cell looking down to the smaller elements in that system is great. Also in a rain forest, Nitrogen molecules and everything they interact with as they move through that system, looking up to larger components, are equally great and yet looking down to the components that amass that system we see the same thing.

Think about that – scale, if only quantifiable by the scale of other systems, is relative. So what could lend to differentiation? Complexity could be an important part of the equation. So what about this – System A is larger and has more components than System B. System B is less complex than System A. If both systems are replicated many times and distributed which may fare better? Hard to say with such a limited theoretical idea but what about Okham’s Razor – the simplest way is the best, essentially. In mechanics, the less moving parts, the less points of failure. A human is a complicated system, a virus is a very simple system and yet a virus can so easily attack the more complicated system. Cells replicate very quickly, and each new cell gets its own copy of the genetic instructions the original parent cell had. I’m rambling but just trying to give some simple examples to think about.

So how do we properly think about scale in systems? What can we learn from successful patterns of scalability that are all around us?

Share/Save/Bookmark

Posted in Software Development | No Comments »

ROO vs Grails – Why both from SpringSource?

Posted by Helena Edelson on March 25th, 2010

Recently I was asked why SpringSource puts out Grails and ROO, what was the difference. Since I have a pretty good idea but wanted to give this person an expert explanation, I asked Ben Alex who pointed me to this post of his.

Share/Save/Bookmark

Posted in Spring ROO | No Comments »

Intalling GIT on Mac OSX

Posted by Helena Edelson on March 13th, 2010

Being a total nube at it, I didn’t realize it was as simple as downloading the GIT installer and running it. Since I’m not into doodleware (i.e. using a ui) I was cloning a GIT repos in seconds.

This is a good guide to simply generating ssh keys: http://help.github.com/mac-key-setup

Share/Save/Bookmark

Posted in Java | No Comments »

Automating The Deployment Process

Posted by Helena Edelson on March 4th, 2010

How many companies fully automate their deploy process? This is not a new idea but I am bringing it up as it is a very important one. I just read and really like the idea bounced by Martin Fowler here, http://martinfowler.com/bliki/BlueGreenDeployment.html, put forth by Dave Farley and Jez Humble. If you think of the SDLC in terms of a manufacturing plant, and around that framework wrap the ideas of LEAN and Six Sigma, not doing this strategy is actually contributing, hundreds of hours per year in high-release companies, to bottlenecks and increase of constraints to flow. The simple idea demonstrated by this sample of blue green deployment shows how simply setting up the proper environment we can easily flip the switch to what instance is production. There are of course many strategies to this, some excellent ones in the cloud and easily transferable to not, particularly when we think in terms of OSGi, but the point remains the same – critical to do for many business justifications, many ways to do it.

Share/Save/Bookmark

Posted in Software Development, Software Development Lifecycle | No Comments »

Simple Asynchronous Processing with Spring’s TaskExecutor

Posted by Helena Edelson on January 27th, 2010

This post is merely meant as a starting guide to tinkering for a light-weight solution to handing off execution of a task for async processing without the overhead of Spring Batch or Spring JMS and Message Brokers, among other middleware solutions.

1. I have a simplistic junit test that merely kicks off the service method to view the path of execution:

/**
 * TaskTests
 *
 * @author Helena Edelson
 * @since v 1.0
 */
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration({"classpath:infrastructure-context.xml"})
public class TaskTests extends BaseTest {
    protected static final Logger logger = Logger.getLogger(TaskTests.class);
    @Autowired private OrderService orderService;

    @Test
    public void testExecution(){
        logger.debug("Starting execution thread...");
        orderService.dispatch(new Order());
    }
}

2. A simple context config:

<?xml version=”1.0″ encoding=”UTF-8″?>
<beans xmlns=”http://www.springframework.org/schema/beans”
xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
xmlns:context=”http://www.springframework.org/schema/context”
xmlns:task=”http://www.springframework.org/schema/task”
xmlns:aop=”http://www.springframework.org/schema/aop”
xmlns:p=”http://www.springframework.org/schema/p”
xsi:schemaLocation=”
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-3.0.xsd”>

<bean id=”taskExecutor” class=”org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor” p:corePoolSize=”5″ p:maxPoolSize=”25″/>

<!– OR alternately: Creates a org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor –>
<task:executor id=”taskExecutor” pool-size=”5-25″ queue-capacity=”100″ rejection-policy=”CALLER_RUNS”/>

</beans>

3. OrderService that delegates to the ThreadPoolTaskExecutor:

/**
 * OrderServiceImpl is used for both orders and returns
 *
 * @author Helena Edelson
 * @MessageEndpoint which is a @Component
 * @since Dec 29, 2009
 */
@Service("orderService")
public class OrderServiceImpl implements OrderService {
    private OrderDao orderDao;
    private MerchantService merchantService;
    private ReceivingService receivingService;
    @Autowired private TaskExecutor taskExecutor;

    @Autowired
    public OrderServiceImpl(OrderDao orderDao, ReceivingService receivingService, MerchantService merchantService) {
        this.orderDao = orderDao;
        this.receivingService = receivingService;
        this.merchantService = merchantService;
    }

    public final void dispatch(final Order order) {
        logger.debug("Starting dispatch execution...");

        if (this.taskExecutor != null) {
            this.taskExecutor.execute(new Runnable() {
                public void run() {
                    executorAsync(order);
                }
            });
        }

        logger.debug("Completed dispatch execution...");
    }

    private final void executorAsync(final Order order) {
        logger.debug("Starting Async execution...");

        daoDatasourceOne.createOrder(order);
        daoDatasourceTwo.createOrder(order);

        logger.debug("Completed Async execution...");
    }

/* Where the output will be: Note the dispatch method returns control to its caller before the async method begins:
2010-01-27 13:23:27,546 [main] DEBUG org.springsource.oms.infrastructure.TaskTests  - Starting execution thread...
2010-01-27 13:23:27,546 [main] DEBUG org.springsource.oms.domain.services.OrderServiceImpl  - Starting dispatch execution...
2010-01-27 13:23:27,546 [main] DEBUG org.springsource.oms.domain.services.OrderServiceImpl  - Completed dispatch execution...
2010-01-27 13:23:27,546 [taskExecutor-1] DEBUG org.springsource.oms.domain.services.OrderServiceImpl  - Starting Async execution...
persisting org.springsource.oms.domain.entities.Order@1f10a67
*/

/**
* Alternately for a different scenario you can play around with this:
*/
public void withExecutor(final Order order) {
        try {
            CompletionService completionService = new ExecutorCompletionService(taskExecutor);

            Object result1 = completionService.submit(new Callable() {
                public Object call() {
                    return daoDatasourceOne.createOrder(order);
                }
            });
            Object result2 = completionService.submit(new Callable() {
                public Object call() {
                    return daoDatasourceTwo.createOrder(order);
                }
            });

            completionService.take().get();
            completionService.take().get();

        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (ExecutionException e) {
            e.printStackTrace();
        }

    }
}

I recommend looking into the @Async annotation, which I will post on shortly. In the meantime here is the ref page for Spring Task and Scheduling: http://static.springsource.org/spring/docs/3.0.x/reference/html/scheduling.html

Share/Save/Bookmark

Posted in Annotations, Concurrency, Java, Spring, Spring Task | No Comments »

Getting Started with Spring Integration

Posted by Helena Edelson on January 20th, 2010

If you have Maven installed:

  1. Download the samples
  2. Unzip them
  3. At the command line, cd into the samples dir and enter: mvn install
  4. In your IDE of choice, set up the project from the root pom

If you are familiar with Enterprise Integration Patterns, here are some of the patterns implemented in each sample:

Pattern / Sample Event Driven Consumer Polling Consumer Message Filter Message Translator Content Based Router Splitter Aggregator Channel Adapter Messaging Gateway Service Activator Request/Reply
cafe X X X X X X X X
filecopy X X X X
errorhandling X X X X
helloworld X X
jms X X X X X
oddeven X X X X X
quote X X X
ws X X X X
xml X X X X X X

From Mark Fisher’s Post:

*NOTE: All of the samples feature certain common patterns that are essential to the underlying Spring Integration core:

  • Message: Spring Integration Messages encapsulate a POJO payload and a header Map (Reference).
  • Message Channel: Spring Integration includes many Message Channel options for both point-to-point and publish-subscribe. Some include queues for buffering while others dispatch directly to subscribers (Reference).
  • Message Endpoint: At a high level, this includes all components that connect to channels for input and/or output.
  • Messaging Mapper: Spring Integration binds inbound Messages to method arguments and method return values to Message payloads and/or headers.
  • Message Dispatcher: In Spring Integration, channels that do not have a queue use Message Dispatchers to invoke their subscribers.
  • Pipes and Filters: This is the most general pattern describing Message-processing components connected in a loosely-coupled way via channels.
  • Message Bus: Spring Integration essentially turns a Spring ApplicationContext into a lightweight Message Bus within which all of these other components are hosted.

Share/Save/Bookmark

Posted in Java | No Comments »

Spring Roo 1.0.0 Released

Posted by Helena Edelson on December 31st, 2009

Just released today!

http://blog.springsource.com/2009/12/31/spring-roo-1-0-0-released/

Share/Save/Bookmark

Posted in Configuration Management, Java, Software Development, Spring, Spring ROO | No Comments »

JMS ActiveMQ Broker Topologies and High-Availability Configuration

Posted by Helena Edelson on December 22nd, 2009

Pure Master Slave BrokerTopology
Pure Master Slave Simplified Topology
I am not actually going to go into Broker topologies, there are many great resources for that such as this by Bruce Snyder: http://www.slideshare.net/bruce.snyder/messaging-with-activemq-presentation or http://activemq.apache.org/topologies.html, all great stuff. This example uses a store and forward topology, or, distributed queues, and incorporates basic authentication:

My use case was to handle down JMS Servers. What I needed to do was implement failover as well as master slave strategies and a topology for message redundancy in case of hardware failure, etc. The client could not have any message loss. With failover, you can see how ActiveMQ switches from the main broker to the second, third, etc on failure. I have a case of four JMS servers in production, each server it is on is load balanced.

There are just a few configurations to add or modify in order to set up JMS Failover with Master/Slave for your broker topology. Here is a basic configuration. For this use case, all JMS servers are configured as standalone versus embedded.

I. Client URI

You will need to add the Failover protocol, either with a basic URI pattern or a composite. In this use case, there are load balanced servers in Production and multiple Development and QA environments which require different configurations for master/slave and failover.

In your application’s properties file for messaging add a modified version of this with your mappings:
activemq.broker.uri=failover://(tcp://localhost:61616,tcp://slaveh2:61616,tcp://master2:61616,tcp://slave2:61616,network:static://(tcp://localhost:61616,tcp://master2:61616,tcp://slave2:61616))?randomize=false

Note: I set connections as locked down (static) communication configurations vs multicast or dynamic discovery so that I know exactly what servers can communicate with each other and how. Also this is assuming you have one set per environment to account for mapping the appropriate IP’s in development, qa, production, dr, etc.

Note: Do not configure networkConnections for master slave, they are handled on the slave with the following configuration:
<masterConnector remoteURI= "tcp://masterhost:61616" userName="wooty" password="woo"/>

II. Spring Configuration

<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory"
destroy-method="stop">
<property name="connectionFactory">
<bean class="org.apache.activemq.spring.ActiveMQConnectionFactory">
<constructor-arg value="${amq.broker.uri}"/>
<property name="userName" value="${activemq.username}"/>
<property name="password" value="${activemq.password}"/>
</bean>
</property
</bean>

III. Broker Configuration

Master

<broker brokerName="{hostname}" waitForSlave="true" xmlns="http://activemq.apache.org/schema/core" dataDirectory="${activemq.base}/data">
<networkConnectors>
<!-- passed in by the client broker URI so you can easily manager per environment: sweet -->
</networkConnectors>

<transportConnectors>
<!-- TCP uses the OpenWire marshaling protocol to convert messages to stream of bytes (and back) -->
<transportConnector name="tcp" uri="tcp://localhost:61616?trace=true" />
<transportConnector name="nio" uri="nio://localhost:61618?trace=true" />
<!-- <transportConnector name="ssl" uri="ssl://localhost:61617"/>
<transportConnector name="http" uri="http://localhost:61613"/
<transportConnector name="https" uri="https://localhost:61222"/> -->
<transportConnectors>
</transportConnectors>

<!-- Basic security and credentials -->
<plugins>
<simpleAuthenticationPlugin>
<users>
<authenticationUser username="system" password="manager" groups="admin, publishers,consumers"/>
</users>
</simpleAuthenticationPlugin>
</plugins>

..more configuration
</broker>

Slave: for ActiveMQ 4.1 or later which also allows for authentication as show below

<broker brokerName="{hostname}Slave" deleteAllMessagesOnStartup="true" xmlns="http://activemq.apache.org/schema/core">
<transportConnectors>
<transportConnector uri="tcp://localhost:61616"/>
</transportConnectors>

<services>
<masterConnector remoteURI= "tcp://masterhost:62001" userName="wooty" password="woo"/
</services>
<!-- Basic security and credentials -->
<plugins>
<simpleAuthenticationPlugin>
<users>
<authenticationUser username="system" password="manager" groups="admin, publishers,consumers"/>
</users>
</simpleAuthenticationPlugin>
</plugins>

</broker>

Share/Save/Bookmark

Posted in ActiveMQ, JMS, JMS Broker, Java, Messaging, Spring JMS | No Comments »

Getting Maven To Compile With Java 1.6

Posted by Helena Edelson on November 28th, 2009

<project>

<build>
<finalName>${groupId}</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
</build>

</project>

Share/Save/Bookmark

Posted in Java, Maven | No Comments »