Java web app integration with PeopleSoft - modularity evolution

A good chunk of my work over the last few years has involved producing Java web applications which work with PeopleSoft.  This post documents the basic module/jar breakdown that I tend to start with and some indication of changes to support future needs. 

The basic war/jars with sample descriptions breakdown like this:

The above works ok in general and has facilitated code sharing between multiple applications.  Some changes have occurred over time resulting in some benefits.  Originally, there were a few API's based on static methods - the most prominent being SQL utilities.  After getting the basic idea working, I converted it to interfaces and implementations.  The resulting benefits included an improvement in testability and multiple concurrent database type support [Oracle and HSQL mainly]. This should have been done using non-static methods initially but at least it was a straight forward conversion in the end.

There are still a few small static API's which make sense with the existing application designs and implementations.  These cover some string and date handling and such which work fine with static method implementations.  There are larger problems to solve instead of worrying about small static utilities.

The above describes a pretty good and time tested design.  There are drivers of change now though that are pushing me to newer and more flexible designs.  The drivers include:

• budget reductions
• lack of adequate staff
• excessively short development time frames
• integrations increasing in quantity and complexity
• increasingly higher application/feature availability

With that said, the general solutions I am pursuing to meet the needs now involve more web services, OSGi and various other messaging/integration technologies.

Now that I am working with OSGi, I am considering it for more general usage at the application level.  My initial plans involved evolutionary changes to existing systems but early prototyping found some issues with that.  I am finding that a number of previous decisions would need to be revisited for existing code bases. That isn't necessarily bad - some types of changes are likely more inline with current general best practices.  In any case where a major impedance mismatch occurs with OSGi; it may be time to reevaluate the implementation or feature. 

The issue with the largest scope so far seems to be due to bundling API interfaces with API implementation in the same jar file.  The OSGi way seems counter to this.  This does affect static API's but it also affects other non-static API's if you didn't plan ahead.  The "best" packaging appears to be separate jar files for API interface and implementations.

Currently, the most annoying issue tied to OSGi is related to the way I handled caching.  I wanted to use a "standard" caching API but JSR 107 was not released as final when I started my cache support.  I ended up using a snapshot of JSR 107 which worked wonderful in several non-OSGi applications.

The use of the snapshot along with some OSGi issues related to Infinispan dependencies resulted in some "irreconcilable differences".  I have yet to find a way to get it working without repackaging something with dummy version info and/or reworking some code.  For now I am putting this support on hold until I have time "to do it right".  There is a newer Infinispan out now which may help (or not) - will figure it out later.  This could be a good time to reevaluate the caching support; most of the caching support was done to handle the fact that in the past the ERP system could barely function normally and external apps/integrations had it near collapse at high utilization times.  Hardware has significantly improved and this hasn't been the case for some time with the ERP. 

Anyways, the expected plan is to start splitting out most existing functionality in OSGi compatible ways. Interfaces and implementations in separate jars and at a more fine grained functionality level. This should be pretty straight forward for large portions of existing code bases;  I suspect the 80/20 rule will apply in this.  I think the initial priority will be some of the utility code which will be useful with some new web services and other integrations.

God bless and Happy coding!
Scott

Caching quagmire

There are a few drastically different views on caching ranging from it is a crutch for poor designs to being a key design aspect which supports scalability in big data environments.  I take a practical view, use what makes the most sense for the problem at hand.

My particular experience involves mostly 2 cases.  An application was created which was did not scale well.  A lack of time to reengineer it left caching as the most expedient solution.  The other case involves an ERP system which wass stretched to the limits on fairly expensive hardware leaving very little capacity for data manipulation by externally integrated applications and integrations.  Limited money combined with substantial growth in the user base led to caching as a way to minimize the impact of non-ERP application overhead. 

Memory tends to be cheaper than large quantities of CPU cores so my organization has a large investment in low/mid-range servers with memory sizes in the 8-32GB+ range.

Where this takes us at the moment is an adhoc collection of solutions which include Memcached, EhCache  and JBoss Infinispan. I find supporting Memcached with a specific Java application of ours overly painful.  Maint on servers or unplanned downtime tends to result in problems with data issues.  I can't blame this fully on Memcached - some developers decided to use Memcached like a DB in some functionality which was a mistake.  EhCache worked OK but I have always found the documentation lacking and trying to figure out feature/licensing aspects between it and Terracotta drove me to find something more straight forward.  The "phone home" feature of EhCache is also a little disconcerting even though there is a way to disable it.  Anyways, I have written a small JSR 107 wrapper allowed me to convert one application from EhCache to Infinispan without any real difficulty.  I can't say that it made any real performance difference but that wasn't the problem which needed solving.  The conversion from Memcached to Infinispan is being planned - that has some challenges due to some of the odd ways it is used.  My preference is to keep the Infinispan cache in-process with the application on each cluster node (at least until I can take a broader look at things and determine if a more grid like environment serving data to our entire application environment makes sense).  I'll have to post updates as the process evolves.

I recently took a quick look at some info on Hazelcast and it has some interesting aspects.  It seemed a little easier to configure (less options and some defaults which may make sense for us).  I may take a closer look at this down the road.