Martin's Blog

Embracing Functional Programming

2011-05-11T09:38:00.000-07:00

Few months ago when learning Scala I ported few personal projects from Java, I wanted to share my experience and discoveries on a particular one, since I thought it demonstrates IMO how I embraced functional programming into my thinking process, applied to something useful. Scala was a good fit for this experiment due to its hybrid [OO+F], so I'll try to depict an implementation transitioning from imperative into functional. I found an old class (good candidate) performing statistical analysis on some collected data, so let's set the stage with summary stats , here's the class definition in the REPL:

Welcome to Scala version 2.9.1.final (Java HotSpot(TM) Server VM, Java 1.6.0_23).
Type in expressions to have them evaluated.
Type :help for more information.
scala>
final class Stats( private val list: List[Double] ) {
    require( list != Nil   , "data list can't be Nil")    
    assert ( !list.isEmpty , "data list can't be empty")
                  
    lazy val data = list.sorted

    lazy val len : Int = data.length

    lazy val sum = data.sum
        
    lazy val min = data.min

    lazy val max = data.max

    lazy val range = max - min
                                                
    lazy val median = if (len%2==0) ((data((len/2)-1) + data((len/2))) /2)
                      else data(((len+1)/2)-1)
                      
    lazy val mean = sum / len
                                                                                
    lazy val variance = {
      @scala.annotation.tailrec
      def vrec(list: List[Double] , tot: Double): Double = list match {
        case head :: tail => vrec( tail , tot + math.pow( head - mean , 2 ) )
        case _ => tot / ( len - 1 )
      }
      vrec(data,0D)
    }

    lazy val standardDeviation = math.sqrt( variance )

    lazy val mode = histogram.maxBy(_._2)._1 

    lazy val histogram = {
       scala.collection.immutable.TreeMap[Double,Int]() ++
       data.groupBy( x => x ).mapValues( _.length )
    }

    def dump { data foreach println }

    override def toString() = {
    new scala.collection.mutable.StringBuilder()
    .append("DATA     : ").append( data )
    .append("\nN        : ").append( len )
    .append("\nMIN      : ").append( min )
    .append("\nMAX      : ").append( max )
    .append("\nRANGE    : ").append( range )
    .append("\nMODE     : ").append( mode )
    .append("\nMEDIAN   : ").append( median )
    .append("\nMEAN     : ").append( mean )
    .append("\nVARIANCE : ").append( variance )
    .append("\nSTRD-DEV : ").append( standardDeviation )
    .append("\nHISTOGRAM: ").append( histogram )
    .mkString
    }             
}
defined class Stats
scala>

let's instantiate our new class with some test data

scala> val stats = new Stats(List[Double](60,10,20,30,20,40,20,50,90,80))
stats: Stats =
DATA     : List(10.0, 20.0, 20.0, 20.0, 30.0, 40.0, 50.0, 60.0, 80.0, 90.0)
N        : 10
MIN      : 10.0
MAX      : 90.0
RANGE    : 80.0
MODE     : 20.0
MEDIAN   : 35.0
MEAN     : 42.0
VARIANCE : 751.1111111111111
STRD-DEV : 27.406406388125955
HISTOGRAM: Map(10.0->1,20.0->3,30.0->1,40.0->1,50.0->1,60.0->1,80.0->1,90.0->1)

so let's then focus on the variance in particular: variance = (1/(N-1) × (the sum of (each data point - mean)²) let's start with the following implementations series, we begin the imperative way with var re-assignment, not FP friendly (but ok since within method scope)

def variance() = {
var sum:Double = 0D
for( x <- data ) {
  sum += math.pow( x - mean , 2 )
}
( sum / ( len - 1 ) )
}

and now with built in map + sum from List impl, this will yield O(n) for map and O(n) for the sum respective collection traversals, not too optimal but better

def variance = data.map( x => math.pow( x - mean , 2 ) ).sum / ( len -1 )

(or use data.view.map() for O(n) traversal -> as explained below)

we move on to a more interesting functional approach with view (could be applied to above one as well), note this time we're forcing a non-strict view on our data list, the new Seq from non-strict-view optimises performance since collection is traversed only once and accumulating the sum without a var, this important 'laziness concept' is something that Stream could achieve:

def variance = {
 val sum = (data.view foldLeft 0D) {
  case (sum,x) => sum + math.pow( x - mean , 2 )
 }
( sum / ( len - 1 ) )
}

now with inner helper function & pattern matching & recursion, calculating on head and adding via recursion, in order to be able to do this I had to define an inner helper function, (note that: recursive calls create a new stack frame and we could potentially reach at some point in time stack-overflow)

def variance = {
def vrec( list:List[Double] ) : Double = list match {
 case head :: tail => math.pow( head - mean , 2 ) + vrec( tail )
 case _ => 0D
}
vrec( data ) / ( len - 1 )
}

and finally (for this particular impl, and falling back to strict since data is not large), the favourite and refined version is optimal since we added an accumulator on the recursive helper signature to sum up our total via tail-recursion (hinting with @tailrec the compiler with optimisation ):

def variance = {
@scala.annotation.tailrec
def vrec(list: List[Double] , tot: Double): Double = list match {
 case head :: tail => vrec( tail , tot + math.pow( head - mean , 2 ) )
 case _ => tot / ( len - 1 )
}
vrec(data,0D)
}

Finally, when comparing the java and scala sources afterwards, it did seem a bit dated to review the imperative way (particularly its redundant verbosity and boilerplate), I don't claim to be an expert on FP nor favour nor force you any specific path, and I'm sure there are more clever ways to do this in FP with syntactic sugar etc, my personal experience and thoughts are subjective, my goal here is to inspire you enough to give it a try yourself (with an open mind) and then you come up with your own conclusions. What I find important to identify here is the fact that when forcing ourselves to think differently (which applies in general terms not just FP, and regardless of language or syntax flavour or complexity at hand) we should always strive for:

Eliminate repetitive duplication: [inspired by DRY & abstraction principles].
Eradicate side effects, specially on unrelated matters, foster immutable things: [inspired by the orthogonal principle].
Design implementations (with intermediate results) that are self-contained, independent, and with a single loose coupled well-defined purpose [inspired by law of demeter].

We must admit though, the polyglot of new JVM languages is raising the bar in language design and engineering, the peril of the ones falling behind is the opportunity for the new ones to blossom, exciting to see languages evolve and improve :-) Till next one, cheers!

CloudFoundry PaaS Test Drive

2011-04-20T21:16:00.000-07:00

I checked out today the recently announced VMWare SpringSource PaaS, I'll have to admit that my first experience went pretty well, from 0 to up & running in the cloud in less than 15 min total (excluding download time from maven for some dependencies, your mileage may vary :), this is amazingly RAD!!. I used the vmc client instead of the STS plug-in, I needed an app to deploy so I used Roo (which also has a plugin) to generate the PetClinic app from sample script file that ships with it, below are my foot-steps, follow along if you happen to be checking it out too:

# install ruby, below is the ubuntu distro way, use pkg installer accordingly
sudo apt‐get install ruby‐full
sudo apt‐get install rubygems
sudo install gem vmc
# either add to $PATH on .bashrc or just create quickly an alias as below
alias vmc='/var/lib/gems/1.8/bin/vmc'
# once installed then launch vmc then with a target cloud + login
vmc target api.cloudfoundry.com
vmc login
> enter email and password, you can use 'vmc passwd' to reset given temp password
> roo
script /samples/clinic.roo
perform package
exit
cd yourRooProjRoot/target/petclinic-0.1.0.BUILD-SNAPSHOT
vmc push
# accept all defaults and give a unique name

https://petclinicroo.cloudfoundry.com And below are some useful vmc app commands you can run afterwards:

vmc -h
vmc apps
vmc stats petclinicroo
vmc logs petclinicroo
vmc instances petclinicroo
vmc logout

I found the vmc client very straightforward to use. Though this is a very plain app test drive, what's important to identify here is (which I found better than other solutions):

Support for some JVM languages

Data storage SQL + NoSQL support [MySQL,MongoDB,Redis], more to come

Open paradigm, no lock in, public, private and cross infrastructure

PaaS is becoming the imminent sweet-spot for application developers, and VMware has stepped up the game in the future of the cloud. Update: 08/10/2011, here's a PoC js data grid doing CRUD ops (some features not implemented) invoking RESTful service using provisioned MongoDB for persistence and Spring profiles
https://restgrid.cloudfoundry.com (System.getenv("VCAP_APPLICATION") != null) ? "Cloud" : "Local" )

public class CloudApplicationContextInitializer implements
              ApplicationContextInitializer<ConfigurableApplicationContext> {
@Override
public void initialize(ConfigurableApplicationContext applicationContext) {
  CloudEnvironment env = new CloudEnvironment();
    if (env.getInstanceInfo() != null) {
      applicationContext.getEnvironment().setActiveProfiles("cloud");
    } else {
      applicationContext.getEnvironment().setActiveProfiles("dev");
    }
  }
}

<mongo:db-factory id="mongoDbFactory"
 dbname="#{serviceProperties['mongodb-db.db']}"
 host="#{serviceProperties['mongodb-db.hostname']}"
 port="#{serviceProperties['mongodb-db.port']}"
 username="#{serviceProperties['mongodb-db.username']}"
 password="#{serviceProperties['mongodb-db.password']}"
/>             

<bean id="mongoTemplate" class="org.springframework.data.document.mongodb.MongoTemplate">
<constructor-arg name="mongoDbFactory" ref="mongoDbFactory"/>
</bean>

<beans profile="dev">
<util:properties id="serviceProperties">
<prop key="mongodb-db.db">db</prop>
<prop key="mongodb-db.hostname">127.0.0.1</prop>
<prop key="mongodb-db.port">27017</prop>
<prop key="mongodb-db.username">xxx</prop>
<prop key="mongodb-db.password">xxx</prop>
</util:properties>
</beans>

<beans profile="cloud">
<cloud:service-properties id="serviceProperties"/>
</beans>

$ vmc apps
+--------------+----+---------+-------------------------------+------------+
| Application  | #  | Health  | URLS                          | Services   |
+--------------+----+---------+-------------------------------+------------+
| petclinicroo | 1  | RUNNING | petclinicroo.cloudfoundry.com |            |
| restgrid     | 1  | RUNNING | restgrid.cloudfoundry.com     | mongodb-db |
+--------------+----+---------+-------------------------------+------------+
$ vmc services
============== System Services ==============
+----------+---------+-------------------------------+
| Service  | Version | Description                   |
+----------+---------+-------------------------------+
| mongodb  | 1.8     | MongoDB NoSQL store           |
| mysql    | 5.1     | MySQL database service        |
| rabbitmq | 2.4     | RabbitMQ messaging service    |
| redis    | 2.2     | Redis key-value store service |
+----------+---------+-------------------------------+
=========== Provisioned Services ============
+------------+---------+
| Name       | Service |
+------------+---------+
| mongodb-db | mongodb |
+------------+---------+

Update 09/2011 I'd like to share a project (that recently deployed to the cloud) from post which plots in a graph histogram for given datasets (you can select multiple rows to view combined results)
https://rest-stats.cloudfoundry.com
And the respective System.getProperties() + System.getenv() could be seen here Cheers

Learning Scala

2011-02-11T15:03:00.000-08:00

I ventured to learn Scala few weeks ago (my new year + long postponed resolution), inspired by its balanced hybrid functional programming and object oriented approach, it's basically like Java on steroids. Scala has so many great features, I can not begin to mention what captivated me from the moment I started reading Martin Odersky's book + book + book . This post is aimed to share my first steps and my discoveries, perhaps it can help you if you happen to be at the early stages as well. Primarily what captured my attention was the fact that you can extend the language, in this post we'll see later on how you can code control structure abstractions, this concept has a lot of potential for exploration! I'll have to admit that my brain is used to think OO the imperative way, as most of us that are Java developers for years, the functional syntax at first bends your mind and tricks your logic, but with time things start blending and sinking in. You'll probably still notice in my code examples I still tend to approach coding with imperative tone, so please forgive my short-comings :) I'd like to begin with this and SBT, here are few simple tasks that I added to my first project, just drop this example under /projroot/project/build/build.scala , and you'll see the tasks when invoking sbt at the shell and then 'actions', the 'meta' task is helpful to dump internal settings such as repos, cp, etc, I really like this tool, simple concise and extendible, you can configure it to manage dependencies and multiple sub projects, etc.

import sbt._ , sbt.Process._ , Process._  
class MySBT(info: ProjectInfo) extends DefaultProject(info) {

val home = Path.fromFile(System.getenv("HOME"))
val libRepo = home / "projs" / "repository" / "lib" ** "*.jar"
//override def unmanagedClasspath = super.unmanagedClasspath +++ libRepo   
//override def mainClass = Some("org.martin.cli.ScellMain")
//override def compileOptions = super.compileOptions ++ compileOptions("-verbose","-optimize") 
override def artifactID = "xxx"

lazy val q    = task { println("bye\n");System.exit(0); None } describedAs("quit")
lazy val env  = task { "env" ! log; None } describedAs("dump env vars")
lazy val ll   = task { "ls -al" ! log; None } describedAs("list all files @ dir")
lazy val fjar = task {
"find project -name *.jar" ! log
None
} describedAs("find project -name *.jar")

lazy val xb = task {
"./bin/scalaxb -d ./xsd -p com.company.sso -v ./xsd/sso.xsd" ! log
None
} describedAs("scalaxb stub generation")

lazy val meta = task {
println("repositories");
repositories.foreach(println)
println("ivyRepositories repositories")
ivyRepositories.foreach(println)
println("compilation options");
compileOptions.foreach(println)
println("compilation CP")
println(compileClasspath)
println("test CP")
println(testClasspath)
println("main external dependencies")
println(mainDependencies.external)
println("main libs dependencies")
println(mainDependencies.libraries)
println("scala dependencies")
println(mainDependencies.scalaJars)
println("unmanagedClasspath")
println(unmanagedClasspath)
None
} describedAs("dump of main meta properties.")

//...more tasks !!
}

Next, I started coding samples in the $scala interpreter snippets from the books, I found this video an excellent and fun resource REPL, so below is a 'script.scala' skeleton for drop and enhance for your needs, it contains few utility you might find useful.

#!/bin/sh
echo "    ________ ___   __   ___     Scala Shell Script"
echo "   / __/ __// _ | / /  / _ |"
echo " __\ \/ /__/ __ |/ /__/ __ |"
echo "/____/\___/_/ |_/____/_/ |_|"
exec scala "$0" "$@"
!#
//////////////////////////////////////////////////////////////////////
lazy val home    = System.getenv("HOME")
lazy val usr     = System.getenv("USERNAME")
lazy val pwd     = System.getProperty("user.dir",".")
lazy val host    = java.net.InetAddress.getLocalHost.getHostName
lazy val props   = System.getProperties().list(System.out)
// props
//////////////////////////////////////////////////////////////////////
for( arg <- args ) println( arg )
//////////////////////////////////////////////////////////////////////
println( usr + "@" + host + ":" + (new java.util.Date()) + ":" + pwd )
//////////////////////////////////////////////////////////////////////
// below is a naive interpreted script for demo
import java.io.File

def files = (new File(pwd)).listFiles

def filesMatching(query: String, matcher: (String, String) => Boolean) = {
for (file <- files; if matcher(file.getName, query)) yield file
}

def filesEnding(query: String)     = filesMatching(query, _.endsWith(_))
def filesContaining(query: String) = filesMatching(query, _.contains(_))
def filesRegex(query: String)      = filesMatching(query, _.matches(_))

def dump( list:List[File] ) { for ( x <- list ) println( x.toURI ) }

dump( filesEnding("*").toList )
dump( filesContaining("scala").toList )
dump( filesRegex(".*").toList )
///////////////////////////////////////////////////////////////////////

Once I run many book samples I began writing a small sample project, here's where I found a lack of tooling in the IDE arena, since I use Eclipse I soon noticed this plug-in is coming along but has few rough edges here and there, I'm sure it's just a matter of time. I find very handy launching from the interpreter os bound commands such as: scala> :sh gedit source_file.scala but that's just a matter of personal preference :) You can take agile approach by launching the interpreter from your IDE, so at your fingertips you can perform REPL for quick snippet PoCs, then pseudo-code a TDD, then code targeted implementations, then materialize tests, then repeat cycle from step#1 etc, continuous building, testing, bug fixing, etc => profit!! David Pollack's book pointed out a very elegant approach with a custom control structure that I adopted in my first project, the capability to augment functionally targeted behaviour, in other words let's look at the below Java sample serialising an obj to the file system, this sample is coded this way to depict the point, notice the convenient closeStream()

public void write( final String file , final Serializable serializable )
throws IOException {
final FileOutputStream     fos = new FileOutputStream( file );
final BufferedOutputStream bos = new BufferedOutputStream( fos );
final GZIPOutputStream     zos = new GZIPOutputStream( bos );
final ObjectOutputStream   out = new ObjectOutputStream( zos );
out.writeObject( serializable );
closeStream( out );
closeStream( zos );
closeStream( bos );
closeStream( fos );
}
public void closeStream( final OutputStream stream ) {
try {
   if ( stream != null )
  stream.close();
 } catch (final Exception e) { }
}

It'd be nice if we could delegate the control to an abstraction via separation of concern so it can do it for us instead, this 'loan pattern' could be handy if consumers of your API could potentially forget to close resources such as tcp /db connections (very typical to see that in poorly written direct jdbc + io etc ): let's see how functional can help here

def streaming[T <:{def close():Unit}, R](target: T)(func: T => R): R =
try {
  func(target)
} finally {
  try {
  target.close()
  println("closed "+target)
 } catch { case _ => }
}

Don't worry too much on the syntax above, we'll review it later, by looking below at read() you'll see the benefits, syntax looks concise and cleaner:

def read[T]( file:String ) : T = {
streaming( new FileInputStream( file ) )   { fis =>
 streaming( new BufferedInputStream( fis ) ){ bis =>
  streaming( new GZIPInputStream( bis ) )    { zis =>
   streaming( new ObjectInputStream( zis ) )  { ois => {
    val t = ois.readObject.asInstanceOf[T]
    println("read " + t)
    t
    }
   }
  }
 }
}
}

def write( file:String , ref:AnyRef ) : Unit = {
require( ref.isInstanceOf[Serializable] , "not @serializable T" )
streaming( new FileOutputStream( file ) )   { fos =>
 streaming( new BufferedOutputStream( fos ) ){ bos =>
  streaming( new GZIPOutputStream( bos ) )    { zos =>
   streaming( new ObjectOutputStream( zos ) )  { out =>
    out.writeObject( ref )
    println("wrote " + ref )
   }
  }
 }
}
}

Let's review the control parameterised signature: def streaming[ T , R ]( target: T )( func: T => R ): R = { .. } type T func(target) returns a type R which R can then be accessed within scope def streaming[T <:{def close():Unit}, R](target: T)(func: T => R): R = {.. } now with a structural type bound on type T that must have public 'close' method in T. Scala allows you to define types based on their structure and pass functions as parameters, which is a very powerful thing.

@serializable
@SerialVersionUID(1234L)
class SerializableType(
   private val id: Int,
 @BeanProperty val name: String,
 @BeanProperty val list: List[Int] ) {
 val tm: Long = System.currentTimeMillis
 override def toString() = {
 tm + " ID: " + id + " - " + name + " : " + list
}
}

object IoDemo {
def main(args: Array[String]) = {
import org.martin.scala.{SerializableType => Pojo} // aliased
val ser:IoDemo = new IoDemo
val file = "/tmp/scala.gzip"
val t = new Pojo(100,"serialization test",List(1,2,3,4,5))
ser.write( file , t )
val tt = ser.read[Pojo]( file )
}
}

Here we have a pojo class with @serializable annotation, one being a List (immutable) collection and a companion object (singleton) with a main() as a poor-man-test, if then compiled and run it yields:

wrote 1297557333613 ID: 100 - serialization test : List(1,2,3,4,5)
closed java.io.ObjectOutputStream@1a786c3
closed java.util.zip.GZIPOutputStream@18088c0
closed java.io.BufferedOutputStream@1922221
closed java.io.FileOutputStream@191d8c1

read 1297557333613 ID: 100 - serialization test : List(1,2,3,4,5)
closed java.io.ObjectInputStream@c832d2
closed java.util.zip.GZIPInputStream@1808199
closed java.io.BufferedInputStream@1bc887b
closed java.io.FileInputStream@a46701

We can now see from output how the streams are closed accordingly in relevant order. Below is another handy one, with conditional logic, appending to a mutable list buffer of T and returning an immutable list of T.

def whiletrue[T]( continue : => Boolean )( in: => T ): List[T] = {
val buffer = new ListBuffer[T]
  while( continue ) buffer += in
buffer.toList
}

Conclusion, imagine taking these concepts even further, combining them with even more abstractions, by now you should have a taste of some functional approach and few of so many great Scala features, I could enumerate some that stand out: type inference, foster immutable state, pattern matching, actors, traits, closures, implicit conversions, and the list goes on. Learning a new language is a quite an experience, you need to step out of your comfort zone and take a whole bunch of new stuff in, after few weeks of reading and coding in Scala, I can tell you that it's been an amazing venture for me, and the journey has just begun.

HMAC(rfc2104) based RESTful service Auth*

2011-01-02T13:39:00.000-08:00

HMAC: Keyed-Hashing for Message Authentication is a good way to assure content from source has not been tampered when it reaches end-point in an un-secured network. I remember implementing many SOAP web services compliant to most OASIS WS-* specs which at the end one could feel the over engineering aspect of it, the soap headers ended up bloated with ws-security-messaging-policy-*, etc. In contrast nowadays with RESTful services there's not much about it due to the nature of being fairly recent in the industry, I kept thinking the other day how one can somewhat use hmac approach but instead of hashing on the check-sum of msg content, instead it would then do it on a predetermined token given by the server, so in this case let's say we're interested more on authenticity than data integrity. Let's imagine we have a RESTful service at hand and you then get a requirement to secure it with a auth-messaging way, eventually having SSL privacy at the transport layer would be ideal scenario combined with this. With a predetermined token which then via a secret shared key would yield a different hash per request which then can be regenerated and verified on the server side. Let's say the secret shared key (in this example) is based on chronological time-stamp + some offset secret computation that both ends know in advance (but you could dictate how that is formulated), then hashing it say via sha-1, that would yield a unique nonce per request which server can even detect brute-force attacks with static token spoofing. This one-off approach is what I wanted to base the experiment on. After reading few sources oauth, wiki, rfc2104, amazon, etc, I then got inspired to start off the new year with this new experiment. First I wanted to make it easy to throw a filter into the chain and let it take care of the matter without touching a line of code my existing rest services, the only impact would be that the client would have to first get a security token prior to accessing services. Here's a chart to depict the flow of execution: Step-3 could be removed or changed, it checks if client source address matches the one generated upon client obtaining token, if consumer comes through say a proxy then same token will be given but then the nonce check will have to be respected to pass check-1, or you could pre-generate tokens per client/ip and look up that list to verify, etc. So the experiment worked just fine, up until I then I created a thin web client to consume and test it, and by thin I mean with html + js, which anybody can right-click and then.....sniff in, so enter code obfuscation + encoding as a way to at least make things harder to reverse-engineer. Here's the sha-1 impl used and upon consumer obtaining /sectoken the server then returns the obfuscated+encoded js which then you could dynamically inject it into to the dom (see ref-impl). Again it's not perfect it but it'll sure take some work to puzzle it together :) Now the filter implementation is based on a ConcurrentHashMap that has an eviction reaper, if a client does not refresh cache access times-tamp it'll evict token and client will have to fetch a token once again, for the most part you could just read the code and follow through, I included the thin html client. One thing I wanted to mention is that upon emulating a brute-force flood of reqs (kinda DoS attack from same source) with a static nonce spoof, the filter detects it and sends an HTTP err, since the salt on client is based on time-stamp (factored by / 500) -> consumer can send up to couple requests per second, server side will serve in discrete mode as long as nonce is unique and incrementally sequential in chronological order from last request, that also causes the hash to be unique per request, below is a screen dump from my fire-bug console displaying some ajax calls, let's take a look: Well, this experiment was fun to implement, as always, take it with a "grain of salt" :) enhance accordingly if taken as is, maybe it inspires you as well or it gives you an idea as it occurred to me, as long as the secret key is unique in your implementation then you'll be ok. Here's the reference implementation archive link. Happy 2011!, cheers.

XSS + SQL Injection Filter

2010-10-31T20:06:00.000-07:00

The dangers are vast from blindly accepting requests these days for any public service that receives requests from clients in the web galaxy out there which could potentially lead to malicious xss | sql | etc injections, last week we started installing mod_security for apache httpd which is an open source intrusion detection that can run embedded or as a reverse proxy. The problem is, what if you do not run httpd in front of your servlet container or jee app server, then you need to have a way to intercept all requests before they reach your application, that's where filters come to the rescue, here's a naive reference implementation filter that wraps the HttpServletRequest object by cloning the parameters state and sanitising them via naive regex token replacement, I recommend reading the OWASP instructions for XSS and SQL there you'll see the top 10 most important rules to follow to avoid becoming a victim of sneaky malicious intents from our widely open and untrustworthy www. I see two approaches here, a pessimistic IDS as intrusion detection (kind of mod_security) or a more optimistic one via content sanitation, take source as a reference implementation and adjust it to your needs, configure the rules with "log" and watch the noise and also the false positives (paranoid rules) and craft then specific regular expressions since I'm sure there'll be new tricky permutations over time. I included the OWASP encoding api calls there for you to investigate, drop me a line with your experiences | discoveries, again this is by no means a full fledged silver bullet but it'll give you an idea on how to approach xss via a filter, security is a huge matter as we know it. I'm kind of troubled to see how much overhead mod_security will add to the farm by pre-processing hundreds of crazy regular expressions on every request (on top of the ones that the load balancer already does prior), seems to me a bit too exhaustive, so we'll just have to find out soon :) Few rules with their corresponding regex, the paranoid ones should always log, once a specific rule+expression behaves as intended then promote it to deny (just make sure you're certain it won't deny on false positive, the tricky part!).


###################################################################################################################
# XSS injections
# html paranoid xss regex base exp (traps lots of false +)
0=log all ^(.*?)(?i)((\\%3C)|\\<)[^\\n]+((\\%3E)|\\>)(.*?)$
# http://www.owasp.org/index.php/HTTP_Response_Splitting
1=log all ^(.*?)(?i)((\\r\\n)+|\\r|\\n|(%0d%0a)+|%0d|%0a)(.*?)$
2=log all ^(.*?)(?i)(\\<|(\\%3C)|&#x73;lt;|&amp;lt;|&lt;|&#60;|&#x3C;)(script|img|%0D)(.*?)$
3=log all ^(.*?)(?i)eval\\((.*)\\)(.*?)$
4=log all ^(.*?)(?i)[\\\"\\\'][\\s]*javascript:(.*)[\\\"\\\'](.*?)$
# SQL-ANSI naive meta keywords injections
#paranoid: ^(.*?)(?i)(delete|grant|insert|drop|replace|truncate|update|create|rename|describe|union|exec|openquery|openrowset|xp_cmdshell|addextendedproc)(.*?)$
5=log all ^(.*?)(?i)[(\\%27)|(\\')]{1}[%20|\\s|:cntrl:]*[(\\%6F)|o|(\\%4F)][(\\%72)|r|(\\%52)]{1}((\\s|\\+|:cntrl:)+).*(delete|grant|insert|drop|alter|replace|truncate|update|create|rename|describe|union)((\\s|\\+|:cntrl:)+)(.*?)$
# MS-SQL-SVR specific meta keywords + hex 0x
6=log all ^(.*?)(?i)(exec|openquery|openrowset|xp_cmdshell|addextendedproc|0x)((\\s|\\+|:cntrl:)+)(.*?)$
###################################################################################################################

Reference source link. Cheers!

Search Capabilities with Lucene

2010-10-07T18:24:00.000-07:00

Last weekend I needed to implement a text search feature, I visited apache foundation and reviewed Solr and Lucene projects, about over a decade ago (geez it's been that long) I remember exploring Inktomi which after the dot.com bubble was acquired by Yahoo, now with Solr (built on top of Lucene) it offers a large array of features just to mention restful capabilities to manage/submit/query/retrieve content, since I wanted direct access and control over it, here's a straight-forward yet full-fledged implementation class which covers some of the most important topics, eventually it's not possible to go over all of them in a blog post but here's my experience I'd like to share.

First, I used the RAMDirectory for in-memory flavor, if you think the index can fit in memory the results will be really snappy, if you have massive content to be index then storing in a file system is the next step, if you run your container on Linux you can use then NIOFSDirectory so then the fs journaling is optimized via NIO (not on MS*s).

Second it's recommendable to always use a single instance of IndexWriter and Searcher over the index, specially if updating on the fly documents/fields, remember to optimize accordingly so then changes are committed.

Third, it is important to mention that QueryParser is not thread-safe.

Fourth, try to keep a single instance of Document | Field, use field.setValue() so then when indexing massive content you avoid GC deallocations of redundant instances.

And last but not least, StandardAnalyzer offers base token digestion but you can implement one as needed to customize the behavior.

Alright then, below are the impl details:

package com.martin.server;

import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.queryParser.QueryParser;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.Searcher;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.store.RAMDirectory;
import org.apache.lucene.util.Version;

public class SearchService {

private static final org.slf4j.Logger log = org.slf4j.LoggerFactory.getLogger(SearchService.class);

private static final String TOKENIZED = "tok";

private RAMDirectory    index = null;
private IndexWriter    writer = null;
private Searcher     searcher = null;
private Analyzer     analyzer = null;
private IndexableDocument doc = null;

public SearchService() { }

@javax.annotation.PostConstruct
public void start() {
try {
index = new RAMDirectory();
// Directory index = NIOFSDirectory.getDirectory("/fs/.."); // NIO on linux :)
analyzer = new StandardAnalyzer(Version.LUCENE_30);
writer   = new IndexWriter( index ,analyzer, true, IndexWriter.MaxFieldLength.LIMITED);// 10k terms max
//writer.setRAMBufferSizeMB(48); //IndexWriter.DEFAULT_RAM_BUFFER_SIZE_MB 16megs find your sweet-spot
//writer.setMaxFieldLength(40000);  // adjust as needed
//writer.setMergeFactor(300);       // adjust as needed
//writer.setMaxBufferedDocs(150);   // adjust as needed in function of setRAMBufferSizeMB
//writer.setMaxMergeDocs(x)         // adjust as needed if reached 2GB limit at 8M set it to 7M
log.info( "--==[ STARTED ]==--");
} catch (final Exception e) {
 log.error("start()", e);
}
}

@javax.annotation.PreDestroy
public void stop() {
try {
   analyzer.close();
   searcher.close();
   writer.close();
   index.close();
   log.info( "--==[ STOPPED ]==--");
} catch (final Exception e) {
  log.error("stop()", e);
}
}

public void buildIndex() {
try {
  writer.commit();
  writer.optimize(true); // block till done
  log.info( "Search.index.mem.size {} bytes"   , index.sizeInBytes()         );
  log.info( "Search.writer.size.buffer {} megs", writer.getRAMBufferSizeMB() );
  writer.close();
  ////////////////////////////////////
  searcher = new IndexSearcher(index);
  ////////////////////////////////////
} catch (final Exception e) {
  log.error("buildIndex()", e);
}
}

public void addIndexableDocument( final String id , final String name , final String content ) {
try {
if ( doc == null ) {
     doc = new IndexableDocument();
}

doc.setFields( id , name , content );

//////////////////////////////////////  
writer.addDocument(doc.getDocument());
//////////////////////////////////////

log.debug("search.index.added {}",doc.toString());

} catch (final Exception e) {
log.error("addDoc()", e);
}
}

public java.util.List<Document> query( final String text , final int max ) {
java.util.List<Document> results = null;
QueryParser parser = null;
try {
  final String txt = QueryParser.escape( java.net.URLDecoder.decode( text ,"UTF-8") ); //sanitize accordingly
  final String qtx = txt+"*"; // (txt + "* OR " + txt + "~"); // etc, craft your query here
  // Note that QueryParser is not thread-safe !!
  parser = new QueryParser(Version.LUCENE_30, TOKENIZED , analyzer );
  // MultiFieldQueryParser(Version.LUCENE_30, String[] fields, analyzer);
  results = doSearch(  parser.parse( qtx ) , max ) ;
} catch (final Exception e) {
  log.error("search()", e);
}
return results;
}

public java.util.List<Document> doSearch( final Query query , final int max ) throws Exception {
 log.debug("search.query {}", query.toString() );
 final java.util.List<Document> results = new java.util.ArrayList<Document>();
  //final TopScoreDocCollector collector = TopScoreDocCollector.create( max , true );  
  ////////////////////////////////////////////////
  //searcher.search( query , collector );
  final TopDocs hits = searcher.search(query,max);
  ////////////////////////////////////////////////  
  log.info("tothits {} , hits.scoreDocs.length {}", hits.totalHits , hits.scoreDocs.length );    
  if ( hits != null && hits.scoreDocs.length > 0 ) {
   for( int i=0; i < hits.scoreDocs.length; i++) {
    // log.debug("explain {} ",searcher.explain(query,hits[i].doc));
    results.add( searcher.doc(hits.scoreDocs[i].doc) ); 
   }
  }      
  return results;
}

// customize as needed
public static class IndexableDocument {
public static final String ID = "i";
public static final String NM = "n";

private final Document doc;
private final Field     id;
private final Field   name;
private final Field    tok;

public IndexableDocument() {
 doc  = new Document();
 id   = new Field(ID, "", Field.Store.YES, Field.Index.NOT_ANALYZED);
 name = new Field(NM, "", Field.Store.YES, Field.Index.NOT_ANALYZED);
 tok  = new Field(TOKENIZED, new java.io.StringReader("") ); // not STORED + ANALYZED
 doc.add(id);
 doc.add(name);
 doc.add(tok);
}

public Document getDocument() {
    return doc;
}
     
public void setFields( final String i , final String n , final String tokens ) {
       id.setValue(i);
       name.setValue(n);
       tok.setValue(new java.io.StringReader(tokens)); // use char[]
}
     
@Override
public String toString() {
    return doc.toString();
}
}

}

And below is a quick test case just to depict usage:

@Test
public void searchQuery() {
try {  

 final SearchService search = new SearchService();  

 search.start();

 for( int i=0; i < 1000; i++) {
 search.addIndexableDocument(String.valueOf(i),"Denim"+String.valueOf(i)    , "denim jeans shorts pant");
 search.addIndexableDocument(String.valueOf(i+1),"Jackets"+String.valueOf(i)  , "jackets collar long sleeve shorts sleeve ");
 search.addIndexableDocument(String.valueOf(i+2),"Underwear"+String.valueOf(i), "underwear bra panties shorts");
 search.addIndexableDocument(String.valueOf(i+3),"Sale"+String.valueOf(i)     , "sale shoes denim jewlery accessories jeans pants shorts jackets watches t-shirts");
 }

 search.buildIndex();

 final List<Document> results = search.query("short",30); 
  
 for( final Document doc : results ) {
    log.info("[ {} ] = {} ",
    doc.get(SearchService.IndexableDocument.ID),
    doc.get(SearchService.IndexableDocument.NM) );
 }

 search.stop();

} catch (final Exception e) {
  e.printStackTrace();
}
}

The id fields you choose to store in the index should be as direct and at minimum as possible, so then you can re-hydrate from cache or DB search results, see IndexableDocument and adjust accordingly based on your needs, wrap that into a restful API returning say json | xml format and voila! there you have it, or simply go with the Solr wrapper if you need more features, such as admin web interface, distributed index management which could be challenging if one thinks of rolling your own, etc ,etc. In a production environment you would have to account for many other important things, yet here's 'in-a-nut-shell' reference implementation.

Till the next experiment!, cheers

Martin

Asynchronous Processing via @Annotation + Dynamic Proxy + ExecutorService

2010-07-31T10:22:00.000-07:00

Hello, I'd like to share an experiment based on a simple asynchronous processing class that acts as a factory and a dynamic proxy to wrap legacy implementation methods that take potentially long execution time blocking from sequential flow into asynchronous non-blocking simplest possible custom refactoring migration.

The challenges encountered here are:

[1] Legacy overloaded method signature + implementations that return a type.

[2] Provide a clean and easy meta annotation to instruct that a method is a potential task to be wrapped into it's own execution thread.

[3] Such meta annotation should also provide a way to register callback user implementations.

[4] Delegate the task to a proxy that handles all implementation details to clearly separate concerns, utilizing: thread pool, concurrent executor with runnable wrappers, etc.

Let's depict a simple legacy implementation scenario:

public interface LegacyDemoInterface {
 public T legacyProcess( final String n , final Long l );
}
// ........
public class LegacyDemoImpl implements LegacyDemoInterface {
 public String legacyProcess( final String n , final Long l ) {
   // usr impl here with long ex tm
   return "x";		
 }
}

So now imagine we could then easily annotate such method like this:

public class LegacyDemoImpl implements LegacyDemoInterface {

 @AsynchProcess(onSuccess="onAsyncSuccess",onFailure="onAsyncFailure")
 public String legacyProcess( final String n , final Long l ) {
  // usr impl here
  return "x";		
 }
 public <T> void onAsyncSuccess( final T result ) {
   System.out.println(">>onAsyncSuccess : " + result );
 }
 public void onAsyncFailure( final Throwable t ) {
  System.err.println(">>onAsyncFailure : " + t.toString());
 }
}

Providing couple callback user implementations where on the asynchronous notification will take place upon success or failure, without using Future<T> but with similar semantics such as @Async from Spring framework, in this case method 'legacyProcess(..)' returns a type(String) which will be passed as a parameter onto success callback. So let's dive into the implementation details, first the @AsynchProcess:

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface AsynchProcess {
 String onSuccess() default "onSuccess";
 String onFailure() default "onFailure";
}

Then the trick is in Dynamic Proxy Factory, note that it has anonymous class approach (on steroids :) to somewhat aprox + emulate the programming concept of 'closure' which in my words I define as: 'the ability to implement embedded functions with the capability for the inner ones to access variables outside of their scope' with corresponding 'final' parameter declarations, there's also an approach to avoid annotating interfaces as part of a proxy nature, in this case we annotated implementation class method shown above to bend some rules.

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class AsyncProcessEngine {

private static final AsyncProcessEngine instance = new AsyncProcessEngine(); //eagerly
private final Logger log = LoggerFactory.getLogger(AsyncProcessEngine.class);
private final ConcurrentHashMap<string,method> methodCache;
private final ExecutorService execService;

private AsyncProcessEngine() {
 execService = Executors.newCachedThreadPool();
 methodCache = new ConcurrentHashMap<string,method>();
 log.info("-=[ AsyncProcessor singleton ]=-");
}

public static AsyncProcessEngine getInstance() {
 return instance;
}

public static void shutdown() {
 instance.methodCache.clear();
 instance.execService.shutdown();
}

public <T> T createAsyncProxyProcess( final Object target , final Class<T> expectedType ) {
return
expectedType.cast(
  Proxy.newProxyInstance(
    target.getClass().getClassLoader() ,
    target.getClass().getInterfaces()  ,
      new InvocationHandler() {
        @Override
        public Object invoke( final Object proxy  ,
                              final Method method ,
                              final Object[] args ) throws Throwable {
       
          final Method implMethod = target.getClass().
          getMethod( method.getName() , method.getParameterTypes() );
                                                                                                
          if ( !implMethod.isAnnotationPresent( AsynchProcess.class ) ) {
               return method.invoke( target , args ); // filter
          }
        
          log.debug("{} method introspection for: {}" , target.toString() , implMethod.toGenericString() );
       
          instance.execService.submit(
             new Runnable() {
               @Override
               public void run() {
                  Method onSuccess = null;
                  Method onFailure = null;
                  try {
                                         
                     final AsynchProcess async = implMethod.getAnnotation( AsynchProcess.class );                                             
                     final String   keySuccess = target.getClass().getName() + target.hashCode() + async.onSuccess();
                     final String   keyFailure = target.getClass().getName() + target.hashCode() + async.onFailure();
                                          
                     if ( !methodCache.containsKey( keySuccess ) || !methodCache.containsKey( keyFailure ) ) {
                       try {
                         onSuccess = target.getClass().getMethod( async.onSuccess() , new Class<?>[] { Object.class    } );
                         onFailure = target.getClass().getMethod( async.onFailure() , new Class<?>[] { Throwable.class } );
                       } catch (final Exception e) { }
                       if ( onSuccess == null || onFailure == null ) {
                          return;
                       } else {
                          methodCache.put( keySuccess , onSuccess );
                          methodCache.put( keyFailure , onFailure );                
                       }
                     } else {
                       onSuccess = methodCache.get( keySuccess );
                       onFailure = methodCache.get( keyFailure );
                     }

                     ////////////////////////////////////////////////////////////////                             
                     try {                             
                        ////////////////////////////////////////////////////////////
                        //invoke target method and pass ret val into success usrImpl
                        onSuccess.invoke( target , method.invoke( target , args ) );
                        ////////////////////////////////////////////////////////////                             
                     } catch (final Throwable t ) {
                        onFailure.invoke( target , t.getCause() );
                     }
                     ////////////////////////////////////////////////////////////////

                 } catch (final Throwable e) {
                   log.error("woha@run()",e);
                 }
            }
          }
          );                    
          return null;
         }
     }     
 )
);
}
}

We could inject via DI proxied objs as shown previously here with a Spring post processor @InjectProxiedHbmTxMgr, but as usual I like keeping things lean and to have control over the behavior that I expect with minimal dependencies possible. Summary: Again, this concept could be taken to so many different levels, as demonstrated it touches very exciting topics :) specially when approaching legacy code refactoring with minimal impact possible, there are other approaches, here's one I thought of. Here's the source archive for download with a sample reference test. Alright then, till the next experiment...cheers!

WebSockets + Netty

2010-01-22T20:18:00.000-08:00

Web applications have evolved significantly the past decade, we've all seen an evolution from thin-client with predominant client/server template-driven paradigms to newer Web2.0 RIA technologies providing smarter clients since now they can do more with plugin run-times.

So far, all requests have been orchestrated from the client 'only', there was no standard connectivity over the network in which the server side could push data down to the client at any point of time.

There's a new revolutionising (incubating) technology: WebSocket, in simple words it provides that key connectivity enhancement with full-duplex (bidirectional) communications. It basically eliminates some of the approaches like inefficient polling or long polling developers have been forced to implement in the past wasting redundant network bandwidth and keeping servers bloated with chatty clients.

Moreover, now that JSR315 / Servlet3.0 is being adopted by the key player server containers (example Jetty continuations) , we can see that very exciting things are evolving on the server side as well, enabling asynch-NIO + eliminating the blocking thread req/con limitation.

Even though specs on both sides are still in draft status, I decided to peek and learn by experimenting on a very simple proof of concept, inspired by few blogs out there, I thought of emulating server broadcasts to client (spawning different threads) simulating somewhat a real time scenario, so here it goes:

First the client, since not all browsers have websocket support yet, I used latest google chrome[4.0.249.43] on my linux ubuntu distribution to test drive this experiment, here's a bare-bone straight-forward html + js client that depicts new features:

<html>
<head>
<title>WebSocket</title>
<script>
var socket;

function openWebSocket() {
  if (window.WebSocket) {
      socket = new WebSocket('ws://127.0.0.1:8080/websocket');
      socket.onopen    = function(event) { alert('WebSocket open!');  };
      socket.onclose   = function(event) { alert('WebSocket closed'); };
      socket.onmessage = function(event) { parse(event.data);         };
  } else {
      alert('Your browser does not support WebSockets yet.');
  }
}

function closeWebSocket() {
 socket.close();
}

function $(id) {
 return document.getElementById(id);
}

function send(message) {
 if (!window.WebSocket) { return; }
 if (socket.readyState == WebSocket.OPEN) {
     socket.send(message);
 } else {
     alert('The WebSocket is not open!');
 }
}

function parse( response ) {
 // parse response: json or xml, etc
 $('out').innerHTML = response;
}

</script>
</head>

<body onload='openWebSocket()' onunload='closeWebSocket()'>
<div id='out'></div>
<!-- more html here -->
</body>
</html>

And on the server side, since I wanted to provide a sample project download I wanted to keep it lean and mean, when learning the new jboss messaging project hornetq I noticed that it is based on netty : client/server NIO framework for network protocols (which by the way its developer has done a great job with a clean impl, kudos! Trustin).

Below is a link to get the project, look for class EnvStatsProcessor which you will see that it spawns 3 threads (if run on linux), each writes data to corresponding channel simulating almost a real time processing per half & 1 sec sleep intervals. The http handshake + upgrade is on class WebSocketServerHandler taken from netty), modify EnvStatsProcessor for your needs, you could enhance it to send json/xml fmt, and client could then use js ajax/dom utilities, note that netty supports binary serializations fmt as well, for that you could use its client api in a rich-client like a swing app and using hessian or google protocol buffers for example.

Here's a quick 30 sec screen-cast to see running: "Look Ma' no polling :)"

(sorry for the low res video, watch in full screen :)

Instructions:

[1] download project (source is included).

[2] extract and cd /mypath/bin

[3] ./run (or .bat if windows)

[4] open in browser http://127.0.0.1:8080

Update 09/16/2011: upgraded netty to 3.2.5.final version and works now on my linux ubuntu 11 distro and also latest ms-*

Cheers.

Hibernate Transaction Management via Dynamic Proxy

2010-01-01T20:07:00.000-08:00

In enterprise applications, transaction management strategy is a very important concept developers need to analyze at early stages of any project architecture cycle.

Inspired by reading book, I started thinking in how to strike a balance in working directly with hibernate but yet reducing all that boiler-plate redundant session/transaction(commit,rollback) code, also taking the less intrusive approach via demarcation similar as spring 3 @Transactional as well as reducing external dependencies lock-in. So here's a custom lab reference implementation sample yet simple transaction proxy, let's revisit few patterns in which we'll cover in this post:

Proxy Pattern: proxy is a wrapper functioning as an interface intercepting targeted object. Decorator Pattern: decorator allows additional behaviour to be added to an existing object dynamically. DAO pattern: data layer/access/persistent operations should be isolated in DAO interfaces + implementations.

Let's begin by looking at a partial sample configuration:

...

...











...

We certainly could use @Autowire or @Inject internally for collaborators but for the sake of simplicity and demonstration purposes we'll leave as is for now to set the stage and later on we'll inject a proxied target, note that we'll foster composition over inheritance in the implementations, now let's take a look at a simple annotation(method target) for our transaction demarcations:

package com.martin.orm.tx.annotation;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface HbmProxiedTransaction { }

Here's a sample interface:

package com.martin.domain.service.dao;
import java.util.List;
import com.martin.domain.service.dto.Profile;
import com.martin.orm.tx.annotation.HbmProxiedTransaction;
public interface SampleInterface {
// note that is not demarcated, usr programmatic hbm approach
public Profile authenticatePrincipal( final String principal, final String pwd );

@HbmProxiedTransaction
public [T] List[T] execSomeQueryHere( final String someCriteriaBy );
}

Note that demarcation takes place at the interface level, this is due to java proxies nature (if you need to proxy a class that has no interface you would need a library like cglib to do so), it could well also be demarcated at the service interface (preferable).

Now here's the proxy implementation (based on org.hibernate.context.ThreadLocalSessionContext configuration)

package com.martin.orm.tx;

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import org.hibernate.SessionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.martin.orm.tx.annotation.HbmProxiedTransaction;

public class HbmTransactionProxyMgr implements InvocationHandler {

private final Logger log = LoggerFactory.getLogger(HbmTransactionProxyMgr.class);

private final Object target;
private final SessionFactory sf;

private HbmTransactionProxyMgr( final SessionFactory sf , final Object target ) {
this.sf  = sf;
this.target = target;
}

public static [T] T
newInstance(final SessionFactory sf, final Object target, final Class[T] expectedType ) {
return expectedType.cast(
  Proxy.newProxyInstance(
     target.getClass().getClassLoader() ,
     target.getClass().getInterfaces()  ,
     new HbmTransactionProxyMgr( sf , target )
   )
 );
}

@Override
public Object
invoke( final Object proxy , final Method method , final Object[] args ) throws Throwable {
if ( proxy == null || method == null ) return null;
Object ret = null;
try {

if ( !method.isAnnotationPresent( HbmProxiedTransaction.class )) {
 return method.invoke( target , args );
}

 sf.getCurrentSession().beginTransaction();
 /////////////////////////////////////
 ret = method.invoke( target , args );
 /////////////////////////////////////
 sf.getCurrentSession().getTransaction().commit();

 log.debug("-=[ tx.proxy ]=-commited()");

} catch (final InvocationTargetException ex) {
 if(sf.getCurrentSession().getTransaction().isActive()) {
    sf.getCurrentSession().getTransaction().rollback();
    log.warn("-=[ tx.proxy ]=-({})rolledback() -> {}" ,
    method.getName(),((ex.getCause()!=null)?ex.getCause().getMessage():ex) );
 }
 throw ex.getTargetException();
} finally {
 if ( ret != null && log.isDebugEnabled() )
 log.debug("-=[ tx.proxy ]=-ret: {}",ret.toString());
}
return ret;
}
}

And here's a new annotation to instruct a tx bean processor for the proxied injection:

package com.martin.orm.tx.annotation;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
@Target(ElementType.FIELD)
@Retention(RetentionPolicy.RUNTIME)
public @interface InjectProxiedHbmTxMgr { }

And the spring bean post processor to inject @InjectProxiedHbmTxMgr

package com.martin.orm.tx;

import java.lang.reflect.Field;
import org.hibernate.SessionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.BeansException;
import org.springframework.beans.FatalBeanException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.context.ApplicationContext;
import org.springframework.util.ReflectionUtils;
import com.martin.orm.tx.annotation.InjectProxiedHbmTxMgr;

public class HbmTransactionProxyProcessor implements BeanPostProcessor {

private final Logger log = LoggerFactory.getLogger(HbmTransactionProxyProcessor.class);

@Autowired
private ApplicationContext ctx;
private final SessionFactory sf;

public HbmTransactionProxyProcessor( final SessionFactory sf ) {
 this.sf = sf;
}

public Object
postProcessBeforeInitialization(final Object bean, final String beanName) throws BeansException {
 return bean;
}

public Object
postProcessAfterInitialization(final Object bean, final String beanName) throws BeansException {

ReflectionUtils.doWithFields( bean.getClass() , new ReflectionUtils.FieldCallback() {
public void doWith(final Field field ) {
 if (field != null && field.isAnnotationPresent(InjectProxiedHbmTxMgr.class) ) {
   final Class type = field.getType();
   final Object dao = ctx.getBean( type );
   if ( dao != null ) {
    try {
      field.setAccessible(true);
      field.set( bean , HbmTransactionProxyMgr.newInstance( sf , dao , type ) ); //!! set
      log.debug("Injected @InjectProxiedHbmTxMgr {}.{} field" , bean.toString() , field.getName() );
    } catch (final IllegalAccessException e) {
      log.error("Injection @InjectProxiedHbmTxMgr for {}.{} field", bean.toString() , field.getName() );
      throw new FatalBeanException(e.getMessage());
    }
   } else {
      log.error("Injection type({}) not found @InjectProxiedHbmTxMgr", field.getType() );
      throw new FatalBeanException(field.getType()+" not found in appContext");
   }
 }
}
});
return bean;
}
}

And finally a snippet of a service impl:

package com.martin.domain.service;
import java.util.List;
import com.martin.domain.service.dao.SampleDaoInterface;
import com.martin.domain.service.dto.Profile;
import com.martin.orm.tx.HbmTransactionProxyFactoryMgr;

public class ServiceImpl implements ServiceInterface {

// for reference processor will inject a proxied obj
@InjectProxiedHbmTxMgr
private SampleDaoInterface dao;

// for reference this dao impl would internally do usr manual/programmatic tx approach
public Profile authenticatePrincipal( final String principal , final String pwd ) {
 return dao.authenticatePrincipal(principal,pwd);
}

// for reference this method is annotated so dao impl would not have boiler-plate code
public [T] List[T] getSomeEntities( final String someCriteriaBy ) throws SomeException {
 return dao.execSomeQueryHere( someCriteriaBy );
}

}

Note that the dao will contain an assigned proxied dao in which interfaces annotated @HbmProxiedTransaction will be wrapped within transaction (commit/rollback) which eliminates tons of boiler-plate + redundant code, also note that if we instead annotate service interface then multiple daos then could participate within a session transaction context.

Summary: This dynamic proxy concept could be applied to many other needs and taken to so many different levels which as demonstrated is based on the decorator pattern in which for instance the AOP concept is based on as well: foster modularization via separation of concerns with the capability to override / intercept / advice targeted behavior, in real production project I would eventually and preferably go with a well known and standard implementation (spring @Transactional, etc) approach for many reasons outside of this scope (if JTA with global transactions, etc), yet this example is just to depict how a dynamic proxy could be applied in an application with a custom local transaction management implementation. Cheers!

Hibernate LazyInitializationException + Flex / BLazeDS AMF Serializations

2009-12-13T11:08:00.000-08:00

On one of my previous posts I shared how I pruned hibernate proxied entities | collections with an annotation directive to avoid lots of DTO handling as a practical approach for prototyping, that is one indirect way of tackling the issue of running across the culprit of LazyInitializationException when reflecting over proxied objects outside of Session context. The concept of configuring Hibernate to perform eager fetching of relationships is really a bad idea specially when object graphs could go too deep with massive n+1 hierarchies, moreover the 'session-in-view' perhaps may be what you not want as well. .

So in this particular instance with BlazeDS/Flex AMF binary serializations and using Spring + Hibernate for persistence I'd like to share one personal approach.

The verdict: you could do pruned DTO by hand or if dealing with raw hibernate objects then: (may not apply to all cases) but try to stay with lazy (default) as much as possible, when there's a need to walk through an object graph do it programaticaly or use 'fetch join' in hql queries as an alternative, if BlazeDS barks here and there with LazyInitException then below is a reference implementation approach that worked ok (when prototyping), so it'll instruct serializers via a custom bean proxy which registers Serializable type (or register more granular types as needed) to filter hibernate proxied beans | persistent collections when the marshaling happens in detach mode outside session.transactional context. You could take it a bit further to utilize Hibernate.initialize(obj) within current session context as it is explained here (look for Initializing collections and proxies).

Cheers!

package com.martin.orm.io;

import java.io.Serializable;
import flex.messaging.io.BeanProxy;
import flex.messaging.io.PropertyProxyRegistry;
import org.hibernate.proxy.HibernateProxy;
import org.hibernate.collection.AbstractPersistentCollection;

public class HbmLazyAmfFilterProxy extends BeanProxy {
private static final long serialVersionUID = 7538263979358789547L;
public HbmLazyAmfFilterProxy() {
 super();
 PropertyProxyRegistry.getRegistry().register(Serializable.class,this);
}

@Override
public Object getValue(Object o, String p) {
 Object val = super.getValue( o , p );
   if ( ( val instanceof HibernateProxy) ||
        ( val instanceof AbstractPersistentCollection ) ) {  
          val = null;
   }
   return val;
 }
}

Or, a more generic bean utility dto reflective populator could be something like this:

//////////////////////////////////////////////////////////////////////////
public void filter( final T orig , final T dest ) throws Exception {
    final PropertyUtilsBean propertyUtilsBean  = new PropertyUtilsBean();  
    final PropertyDescriptor[] origDescriptors = 
          propertyUtilsBean.getPropertyDescriptors(orig);
        for (int i = 0; i < origDescriptors.length; i++) {
            final String name = origDescriptors[i].getName();
            if ("class".equals(name)) continue;                          
            if ( propertyUtilsBean.isReadable ( orig , name ) &&
                 propertyUtilsBean.isWriteable( dest , name ) ) {
                 try {
                   final Object value = 
                         propertyUtilsBean.getSimpleProperty( orig , name );
                   if ( ( value instanceof HibernateProxy) || 
                        ( value instanceof AbstractPersistentCollection ) ) { 
                          continue; // filter
                   }                    
                   BeanUtils.copyProperty( dest , name , value );
                 } catch (final NoSuchMethodException e) { 
                   // deal with it
                 }
            }
       }
}

Thread-safe MapCache with Reaper

2009-12-07T19:51:00.000-08:00

Sometimes when writing middle-ware applications there's a need for a simple thread safe parameterized mem map cache with eviction reaper, generally non locking on retrieval and with internal wrapper class. By reading the latest java.util.concurrent api doc there is a rich list of collections built with concurrency in mind, I could not find one with an eviction reaper built-in so today I assembled one and here's a reference implementation I'd like to share.

package com.martin.cache;

import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import java.io.Externalizable;
import java.util.Map;
import java.util.Set;
import java.util.HashSet;
import java.util.Iterator;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ScheduledThreadPoolExecutor;
 
/**
 * Thread safe simple parameterized map cache with eviction reaper, 
 * generally non locking on retrieval + internal <V> wrapper class.
 * @author MartinZ 12/07/2009
 * @param K key type
 * @param V value type
 */ 
public final class MapCache< K , V > {
   
  private class Reaper implements Runnable {
    public void run() {
      evict();
    }
  }

  public interface EvictionListener< K > {
    void evicted( K key );
  }
     
  /////////////////////////////////////////////////////////////////////
  public static final long DEFAULT_EVICT_INTERVAL_MS = (30 * 1000);
  public static final long DEFULT_15MINUTES_CACHE_MS = (15 * 60 * 1000); 
  ///////////////////////////////////////////////////////////////////// 
  
  private final ConcurrentHashMap< K , Value<V> > map;
  private final ScheduledThreadPoolExecutor timer;
  private final HashSet<EvictionListener<K>> listeners;
    private final long interval;
    
  public MapCache() {
      this(DEFAULT_EVICT_INTERVAL_MS);
  }
  
  public MapCache( Long tm ) {
    interval  = (tm==null)?DEFAULT_EVICT_INTERVAL_MS:tm;
    map       = new ConcurrentHashMap<K,Value<V>>();
    timer     = new ScheduledThreadPoolExecutor(1);
    listeners = new HashSet<EvictionListener<K>>();
  }
 
  public void addEvictionListener(EvictionListener<K> l) {
    listeners.add(l);
  }
  public void removeEvictionListener(EvictionListener<K> l) {
    listeners.remove(l);
  }

  public int getSize() {
    return map.size();
  }

  public void clear() {
    map.clear();
  }
  
  public void start() {
    timer.scheduleWithFixedDelay(
       new Reaper(),0,interval,TimeUnit.MILLISECONDS
    );
  }

  public void stop() {
    timer.shutdown();
  }

  /**   
   * @param key != null
   * @param val != null
   * uses default 15 minute time to live since last get   
   */  
  public V put( K key , V val ) {
    return put( key , val , DEFULT_15MINUTES_CACHE_MS );
  }  
  
  /**   
   * @param key != null
   * @param val != null
   * @param time ms to keep an entry alive. 0 = never evict.    
   */  
  public V put( K key , V val , long tm ) {
    if(key == null || val == null ) return null;
    Value<V> value  = new Value<V>( val , tm );
    Value<V> retval = map.put( key , value );
    return (retval != null ? retval.value : null);
  }

  public V get(K key) {
    if(key == null) return null;
    Value<V> val = map.get(key);
    if(val == null) return null;    
    val.refreshTimestamp();
    return val.value;
  }

  public Value<V> getEntry( K key ) {
    if(key == null) return null;
    Value<V> val = map.get(key);
    if(val == null) return null;
    val.refreshTimestamp();
    return val;
  }

  public V remove(K key) {
    Value<V> val = map.remove(key);
    return (val != null ? val.value : null);
  }

  public Set<Map.Entry<K,Value<V>>> entrySet() {
    return map.entrySet();
  }
 
  private void evict() {
    for( Iterator<Map.Entry<K,Value<V>>> it = 
       map.entrySet().iterator(); it.hasNext();) {
       Map.Entry<K,Value<V>> entry = it.next();
       Value<V> val = entry.getValue();
       if(val != null) {
        if( (val.timeout > 0) && 
            (System.currentTimeMillis() > 
              (val.insertion + val.timeout))) {
            it.remove(); // rm from iterator
            notifyListeners( entry.getKey() );
        }
       }
    }
  }

  private void notifyListeners( K key ) {
    for(EvictionListener<K> l: listeners) {
      try { l.evicted(key);
      } catch(Throwable t) { }
    }
  }

  public String toString() {
    StringBuilder sb=new StringBuilder();
    for(Map.Entry<K,Value<V>> entry: map.entrySet()) {
      Value<V> val=entry.getValue();
      sb.append(entry.getKey()).append(": ");
      sb.append(entry.getValue().getValue());
      sb.append(" (expiration tm.ms: ");
      sb.append(val.getTimeout()).append(")\n");
    }
    return sb.toString();
  }  
  
    ///////////////////// internal wrapper ////////////////////  
  public static class Value< V > implements Externalizable {
    private V value;
    private long timeout;
    private transient long insertion=System.currentTimeMillis();
    private static final long serialVersionUID = 1100000000001L;
    
    public Value( V value , long timeout ) {
      this.value   = value;
      this.timeout = timeout;
    }

    public V getValue()            { return value;     }
    public long getTimeout()       { return timeout;   }
    public long getInsertionTime() { return insertion; }
    
    public void refreshTimestamp() {
      if( timeout > 0 )
      insertion=System.currentTimeMillis();
    }
    
    public void writeExternal(ObjectOutput out) 
    throws IOException {
      out.writeLong(timeout);
      out.writeObject(value);
    }

    @SuppressWarnings("unchecked")
    public void readExternal(ObjectInput in) 
    throws IOException,ClassNotFoundException {
      timeout   = in.readLong();
      value     = (V) in.readObject();
      insertion = System.currentTimeMillis();      
    }
  }
}

Distributed computing concepts

2009-11-15T14:09:00.000-08:00

Whether load-balancing, fail-over, replication, distributed FS, cloud capabilities, etc, for clustering application servers nowadays, the ultimate goal is to improve quality of highly available + scalable service with less latency and fault tolerance features. By looking under the hood on how J2EE servers do it, I came across JGroups library which is the toolkit JBoss uses per instance. JGroups is a library for reliable multicast communication, it provides a flexible protocol stack which allows developers to adapt to specific application requirements and specific network topology characteristics, its main features:

Transport protocols: UDP (IP Multicast), TCP, JMS
Fragmentation of large messages
Reliable unicast and multicast message transmission. Lost messages are retransmitted
Failure detection: crashed members are excluded from the membership
Ordering protocols: Atomic (all-or-none delivery), Fifo, Causal, Total Order
Membership
Encryption

With this lower level reliable communication library, developers could then implement sophisticated middle-ware applications with distributed computing based features, just to mention couple ones that come to my mind: memcache ( distributed map ), map-reduce / fork-join model for task execution in a private cluster/cloud, and many more capabilities, here's a partial reference source implementation that I experimented with:

// create
org.jgroups.JChannelFactory factory = new JChannelFactory("udp.xml"); //tcp.xml etc
org.jgroups.JChannel channel = (JChannel) factory.createChannel();
channel.setOpt( Channel.AUTO_RECONNECT , true );
channel.setOpt( Channel.AUTO_GETSTATE  , true );
...
// start
channel.connect( "CLUSTER-NAME" );
org.jgroups.blocks.PullPushAdapter adapter = new PullPushAdapter(channel,this,this);
channel.getState(null,10000); // 10 secs conn tolerance
...
// stop
channel.disconnect();
channel.close();

// simple serialized user defined type msg send
ClusterMsg m = new ClusterMsg( action );
byte[] buffer = org.jgroups.util.Util.objectToByteBuffer( m );
Message msg = new Message( dest , src , buffer , 0 , buffer.length );
channel.send( msg );

// from Interface org.jgroups.MessageListener
public byte[] getState() { ... }
public void receive( Message msg ) {
Serializable s = (Serializable)
    org.jgroups.util.Util.objectFromByteBuffer(msg.getBuffer());
    if( s instanceof ClusterMsg ) {
       ClusterMsg cmsg = (ClusterMsg) s;
       switch( cmsg.getAction() ) {
        // do something
       }
    }
}
public void setState( byte[] newState ) { ... }

// from Interface org.jgroups.MembershipListener
public void viewAccepted( org.jgroups.View currentView ) { ... }

parallelism pseudo-code concept [divide-conquer | map-reduce | fork-join] Result solve(Problem problem) { if (problem is small enough) result = problem.solve() else { split problem into independent parts fork new subtasks to solve each part join all subtasks compose result from sub-results } } In general, split tasks could then be allocated | distributed to available nodes for execution, the middle-ware could be able to perform a network node pre-inspection to determine its hardware capabilities of each node corresponds for computing or data storage based on their processor, memory, HD power, etc. Thus depending on what a certain task does, allocation should be in function of what a certain node can provide at its best. Here's a static img for illustration of my simple PoC experiment, I'll spawn 3 instances (same host) with one being the master node, and two slave ones, there's a simple map being serialized and acting as a simple memcache implementation as well as a simple distributed task execution capability, I created a simple web UI which shows all nodes in cluster along with the protocol stack configured, and one can run tasks, send sessages to all nodes and shut-down a targeted node or all cluster in view: Hard to see it all working from img, but the main concept is there and functional, now the cool thing is that if either node goes down (including master one) the whole cluster is aware of it and the state is being serialized and distributed upon node bootstrap via the org.jgroups.MessageListener. No need to reinvent the wheel here, as we know most containers have clustering built-in and there are already middle-ware open source projects: GridGain , Hadoop and other products in the arena such as: Coherence , Terracota , etc. Finally, with the new explosion of cloud services such as bottom up: IaaS, PaaS, SaaS with flexible 'pay as you need' on demand features, we can see that there's a new trend to provide cloud services in which under the covers it is dynamic provisioning of distributed computing platform at its core.

Windows installer for a java swing app

2009-09-18T11:53:00.001-07:00

Early this year I contracted a very small project where the desktop java swing app needed to be installed and run on MS-OS(s), since I used a cross-platform java executable wrapper: launch4j to generate an .exe I could then automate an installer with features like: un-installer, detect core dependencies such as JRE and install it not present as well as Adobe reader, registry settings, readme file, logo splash, etc, etc, and more. Simple cmd compiler invokation as: makensis.exe /V4 InstallScriptSample.nsi Resolution: NSIS did the job, scriptable installer with rich set of options. Below is an example configuration I used, take it as a guideline and modify accordingly for your needs. Cheers!

;----------------------------------------------------
; File: InstallScriptSample.nsi
; Installer using -> http://nsis.sourceforge.net
;----------------------------------------------------
; Note replace "MyApp" with desired app name
;----------------------------------------------------

SetCompressor /SOLID lzma

!include "MUI.nsh"
!insertmacro MUI_PAGE_WELCOME
#!insertmacro MUI_PAGE_DIRECTORY
#!insertmacro MUI_PAGE_COMPONENTS
!insertmacro MUI_PAGE_INSTFILES
!insertmacro MUI_UNPAGE_CONFIRM
!insertmacro MUI_UNPAGE_INSTFILES
!define MUI_FINISHPAGE_NOAUTOCLOSE
!define MUI_FINISHPAGE_RUN
!define MUI_FINISHPAGE_RUN_CHECKED
!define MUI_FINISHPAGE_RUN_TEXT "Start MyApp"
!define MUI_FINISHPAGE_RUN_FUNCTION "LaunchLink"
!define MUI_FINISHPAGE_SHOWREADME_CHECKED
!define MUI_FINISHPAGE_SHOWREADME $INSTDIR\README.txt
!insertmacro MUI_PAGE_FINISH

;----------------------------------
!insertmacro MUI_LANGUAGE "English"
;----------------------------------

#Page directory
#Page components
#Page instfiles

;--------------------------------
; Main Install settings
;--------------------------------

Name "MyApp"
Caption "MyApp Installer | Please wait!"
Icon "icon.ico"
#InstallDir "$PROGRAMFILES\MyApp"
InstallDir "c:\MyApp"
OutFile "Setup.exe"
ShowInstDetails show
ShowUninstDetails show
AutoCloseWindow true
RequestExecutionLevel admin
BrandingText "MyApp v1.0"
AllowRootDirInstall false
InstallColors 00FF00 000000
InstallDirRegKey HKCU "Software\MyApp" ""
InstallDirRegKey HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\MyApp" "UninstallString"

#---------------- installer section start ----------------
section

Call Includes
Call CheckJRE
Call InstallJre
Call InstallAdobe

Success:
Call CreateShortCutsAndUninstaller
Goto Done

Done:

# messageBox MB_OK "Installation was successful @ $INSTDIR $\r$\n \
# Please use desktop shortcut [MyApp] to start-up application."

sectionEnd

#---------------- installer section end --------------------


#---------------- uninstaller section start ----------------

section "uninstall"
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\MyApp" "DisplayName" ""
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\MyApp" "UninstallString" ""
DeleteRegKey /ifempty HKCU "Software\MyApp"
delete "$DESKTOP\MyApp.lnk"
delete "$STARTMENU\MyApp.lnk"
delete "$SMPROGRAMS\MyApp.lnk"
rmdir /r $INSTDIR

sectionEnd

#---------------- uninstaller section end ------------------

Function LaunchLink
ExecShell "" "$INSTDIR\MyApp.exe"
FunctionEnd


;--------------------------------
Function Includes

WriteRegStr HKCU "Software\MyApp" "" $INSTDIR

SetOverwrite on

SetOutPath $INSTDIR

File run.cmd
File MyApp.exe
File README.txt
File QA.txt

SetOutPath $INSTDIR\config
File .\config\config.xml
File .\config\hbm.cfg.xml
File .\config\log4j.xml

SetOutPath $INSTDIR\db
File .\db\HSQLDB.app.log
File .\db\HSQLDB.backup
File .\db\HSQLDB.data
File .\db\HSQLDB.properties
File .\db\HSQLDB.script

SetOutPath $INSTDIR\img
File /r ".\img\*.*"

SetOutPath $INSTDIR\lib
File /r ".\lib\*.*"

SetOutPath $INSTDIR\rpt
File /r ".\rpt\*.*"

SetOutPath $INSTDIR\inc
File .\inc\jre-6u11-windows-i586-p.exe
File .\inc\AdbeRdr90_en_US.exe

SetOverwrite off

FunctionEnd

;--------------------------------

Function InstallJre

ExecWait '$INSTDIR\inc\jre-6u11-windows-i586-p.exe /quiet' $0

FunctionEnd

;--------------------------------

Function InstallAdobe

ExecWait '$INSTDIR\inc\AdbeRdr90_en_US.exe /sAll /rs' $0

FunctionEnd

;--------------------------------

Function CreateShortCutsAndUninstaller

#    push $0
#	FileOpen $0 $INSTDIR\MyApp.cmd w
#	FileWrite $0 "cd $INSTDIR\bin$\r$\n"
#	FileWrite $0 "run.cmd$\r$\n"	
#	FileClose $0
#    pop $0

CreateShortCut "$DESKTOP\MyApp.lnk" "$INSTDIR\MyApp.exe" \

"" "$INSTDIR\img\icon.ico" 0 SW_SHOWNORMAL CONTROL|SHIFT|F5 "MyApp"



CreateShortCut "$SMPROGRAMS\MyApp.lnk" "$INSTDIR\MyApp.exe" \

"" "$INSTDIR\img\icon.ico" 0 SW_SHOWNORMAL CONTROL|SHIFT|F5 "MyApp"



CreateShortCut "$STARTMENU\MyApp.lnk" "$INSTDIR\MyApp.exe" \

"" "$INSTDIR\img\icon.ico" 0 SW_SHOWNORMAL CONTROL|SHIFT|F5 "MyApp"



WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\MyApp" "DisplayName" "MyApp (Uninstall)"

WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\MyApp" "UninstallString" "$INSTDIR\Uninstall.exe"



writeUninstaller "$INSTDIR\Uninstall.exe"



FunctionEnd



;--------------------------------

Function CheckJRE

push $0

push $1


ClearErrors

ReadRegStr $0 HKLM "SOFTWARE\JavaSoft\Java Runtime Environment" "CurrentVersion"

ReadRegStr $1 HKLM "SOFTWARE\JavaSoft\Java Runtime Environment\$0" "JavaHome"

StrCpy $1 "$1\bin\java.exe"

pop $0

exch $1

FunctionEnd

;--------------------------------
; eof
;--------------------------------

Cross-platform java executable wrapper

2009-09-18T11:50:00.000-07:00

Early this year I needed to wrap a small project java swing app into a native MS-OS executable, after deliberating on few products out there I chose launch4j Resolution: It did the job, it has easy to use streamlined UI which generates an xml configuration file which then you can reuse to rebuild exe. Below is an example configuration I used, take it as a guideline and modify accordingly for your needs. Cheers!




true
gui
lib\org.myapp_1_0_0.jar
C:\tmp\MyApp.exe
Java Runtime Error
config/log4j.xml config/config.xml
.
normal
http://java.com/download
http://java.sun.com
false
false

icon.ico

MyApp
MyApp


org.myns.myproj.MyAppMain
lib/org.myapp_1_0_0.jar
lib/commons-collections-3.1.jar
lib/commons-io-1.4.jar
lib/commons-lang.jar
lib/commons-logging-1.1.jar



C:\Program Files\Java\jre6
1.6.0_11

preferJre
128
256


img\logo.bmp
true
60
true


An error occurred while starting the application.
This application was configured to use a bundled Java Runtime Environment but the runtime is missing or corrupted.
This application requires a Java Runtime Environment
The registry refers to a nonexistent Java Runtime Environment installation or the runtime is corrupted.
An application instance is already running.

JVM Agent - Class Spy Utility

2009-08-31T21:23:00.001-07:00

I've been experimenting recently on a PoC java agent utility which dumps high & low level details for any class loaded by JVM at any point of time. It does not perform any bytecode instrumentation manipulation / alteration but rather loads all classes available / loaded by the JVM in context into a UI in which it organizes classes by package namespace into a hierarchical tree structure. This utility comes handy when you need to introspect class bytecode blueprint signatures that may be in question at runtime, similar to the javap utility that ships with jdk. According to the specs the agent jar must contain:

MANIFEST.MF
...
Agent-Class: org.martin.agent.jvm.JvmAgent
Premain-Class: org.martin.agent.jvm.JvmAgent
Boot-Class-Path: jvmagent_1_0_0.jar
Can-Redefine-Classes: false
Can-Retransform-Classes: false

(note that redefinitions + transformations are not set since neither is needed here) and method premain (similar to a 'main') signature hook is as follows: org.martin.agent.jvm.JvmAgent public static void premain(String agentArgs, Instrumentation inst) { ... } once we have instrumentation access we can then: [1] register a custom class transformer

instrumentation.addTransformer( new MyClassLoadTransformer() );
...
class MyClassLoadTransformer implements ClassFileTransformer {
public byte[] transform(
ClassLoader loader,
String fullyQualifiedClassName,
Class classBeingRedefined,
ProtectionDomain protectionDomain,
byte[] classofileBuffer) throws IllegalClassFormatException {
// illustration here!
}
}

[2] and invoke the instrumentation method: final Class[] clsArr = instrumentation.getAllLoadedClasses(); which feeds the UI by representing pkgs hierarchicaly in a tree: Once a desired class node is selected, it prompts for high / low level detail?, either selection will dump its details on to System.out, below is a console sample of output: Instructions: [1] download jar (only 22+k) and place it either in /lib or /bin of your app [2] Then you just need to add -javaagent parameter such as: java -javaagent:jvmagent_1_0_0.jar ...(more options here) (depending on where it is : relative path -javaagent:../lib/jvmagent_1_0_0.jar , so configure accordingly!) Agent will launch and load a UI with an initial draw, if your bootstrap process takes a bit longer then at any point you can press refresh button to re-load jvm state, the jvm + your application will be dynamically loading on demand classes based on the dependencies needed, so refresh accordingly. Bytecode class dump disassembler credits. Conclusion Now imagine taking this concept further, profilers per say, or by using bytecode manipulation libraries such as asm or javassist an example is javarebel which allows reloading classes dynamically at runtime (even if making significant changes) without needing to restart container. I do have a working version agent where does hotswap (as long as new bytecode respects same signature and only changes within method implementation), however JVM can become very stale if you violate that limitation (at least up until now in jdk 6+). Update 12-27-2010: There's a new VM that allows unlimited class redefinition at runtime http://ssw.jku.at/dcevm Till next time, cheers!

Java shell

2009-06-23T16:34:00.001-07:00

I've been having fun writting a utility java shell, I use cygwin on windows but I needed something more integrated with java such has building class paths, invoking java/ant/groovy /os scripts, scanning in context class loader certain classes | @Annotations, etc, etc. And also when I switch to Linux there it is for me as well. Some basic commands are os file system handling such as ls / rm / mk / grep / etc.

Internals use java.io.Console and a custom annotation:

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface Command {
String name()        default "";
String description() default "";
String abbrev()      default "";
String def()         default "";
}

So anytime I need a new command I can annotate it like below and instantly it becomes available via method reflection invocation:

@Command(name="cmd", description="sample cmd", abbrev=".", def="?")
public void cmd( String str ) throws Exception {
...
}

Here's an example below for few common file system cmds: ls , lsr (just like ls but with recursive capabilities building a tree)

Note that commands can be chained 'piped' taking res output as input like in *nix such as:

lsr src | grep '06-11'

And also io redirect '>' result:

ls . | lsr src | grep '06-11' > result.log

Or io redirect appended '>>'

ls . | lsr src | grep '06-11' >> result.log etc I'll have to admit that grep command which takes regular expressions to search is my favorite command of them all (specially in windows cuz u know y), it's so powerfull when combined with others.

Here's a list of the help for all cmds implemented so far (keeps growing by the day) Today I included a disassembler via 'javap' which scans in context class loader via a package prefix and shows public class signatures, it comes handy when you need to introspect bytecode loaded and being currently invoked. It also can open a script file and sequentially execute cmds but for that there's a groovy cmd which I can invoke and run groovy scripts with dyn-language richness, one example is I run [def ant = new AntBuilder()] via groovy ;-).

Now one thing I hate to do is walking by hand on file system typing long paths, so I added a simple 'fs' cmd which launches a swing UI which facilitates file system operations, I hooked most common cmds in a menu and results are directed to console:

I know it looks like a toy but it surely comes handy on day by day con$ole operations, heck I don't know what I'll add next but it sure saves the day, I love being productive with tools that do things my way ;-) and I can easily throw a new feature at it in a matter of minutes. One final trick (for windows) is that you can invoke it directly from windows file explorer anytime you right-click on a folder (shows on menu): file: reg.reg

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Directory\shell\jell]
@="jell"
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Directory\shell\jell\command]
@="cmd.exe /k \"cd %L\"&jell"

I'll probably pack it once it becomes stable so I can share as GA, that is if anybody finds this useful, Well, till next time, cheers...

Ant + Ivy > Repository +

2009-06-12T20:42:00.001-07:00

I'd like to share my Ivy configuration that I use to build my Java repository and to manage transitive dependencies / conflicts, first I'd like to point out that I prefer controlling internals, the 'convention over configuration ease' which by the way is a great concept but seems to fall short sooner or later when it comes to opening the hood and morphing things, I like simplicity and once a tool works for you the way you want then leave it as is. I've read many posts regarding Ant <> Maven which at the end of the day the reality is that there's no such a thing as a 'silver bullet' when building complex + modularized projects. I think that: "it's not just what a tool can do for you magically, but you can do 'with' the tool in a harmonious/flexible/extensible way" ;-) I guess that's what I love most about Spring framework, BTW both mentioned products are great, evaluate what you're getting into [pros/cons] and choose wisely. My choice = ubiquitous Ant + Ivy cuts it for me.


  
   
    This is the main dependency repository list.

Ant (partial) snippet:

Cheers.

AppEngine & BlazeDS (Messaging)

2009-05-10T08:47:00.000-07:00

http://riaspringblaze.appspot.com https://riaspringblaze.appspot.com

Hello!

I added a simple messaging test drive to existing flex PoC app, this time I updated Spring version from 2.5.5 to 3.0.0M2 and from flexM1 to flex-1.0.0.RC1 with my already existing 'slight-hacked' core.jar from my previous blog , it went pretty well and here are the results:

Note that Spring changed @ web-application-config.xml to new 'remoting-destination' ->

Here's a snippet flex client set up ->

And finally this is my new list of libraries/dependencies deployed to the cloud for your reference->

martin@martin-ubuntu9 ~/projs/blazeappengine/war/WEB-INF/lib
$ ls -1
antlr-3.0.1.jar
aopalliance.jar
appengine-api-1.0-sdk-1.2.0.jar
asm-3.1.jar
aspectjrt.jar
backport-util-concurrent.jar
cfgatewayadapter.jar
cglib-2.2.jar
commons-beanutils.jar
commons-collections-3.1.jar
commons-fileupload-1.2.1.jar
commons-lang.jar
commons-logging-1.1.jar
concurrent.jar
datanucleus-appengine-1.0.0.final.jar
datanucleus-core-1.1.0.jar
datanucleus-jpa-1.1.0.jar
flex-messaging-common.jar
flex-messaging-core.jar
flex-messaging-opt.jar
flex-messaging-proxy.jar
flex-messaging-remoting.jar
geronimo-jpa_3.0_spec-1.1.1.jar
geronimo-jta_1.1_spec-1.1.1.jar
jackson-core-asl-0.9.9-6.jar
jdo2-api-2.3-SNAPSHOT.jar
org.springframework.aop-3.0.0.M2.jar
org.springframework.aspects-3.0.0.M2.jar
org.springframework.beans-3.0.0.M2.jar
org.springframework.context-3.0.0.M2.jar
org.springframework.context.support-3.0.0.M2.jar
org.springframework.core-3.0.0.M2.jar
org.springframework.expression-3.0.0.M2.jar
org.springframework.flex-1.0.0.RC1.jar
org.springframework.transaction-3.0.0.M2.jar
org.springframework.web-3.0.0.M2.jar
org.springframework.web.servlet-3.0.0.M2.jar
xalan.jar

Give it a try, subscribe to pre-determined dest+topic and emulate a msg push from server, you'll see there an asynch msg received by the client consumer, better yet launch 2 or 3 browsers, subscribe from all and send msgs from only one browser isntance, you'll see all consumers being published since they all subscribe to same predetermined topic.... ;-)

Cheers.

Bean @Filter Annotation

2009-05-08T08:40:00.000-07:00

Hello!

When performing introspections through your own reflective or 3rd party library and specially if your objects are 'proxied' within a certain context, sometimes there's a need to instruct reflective libraries to 'filter' certain class memebers somewhat similar to intrinsic 'transient' keyword (serializations), in this case I thought of a non-intrusive way to annotate with metadata a runtime directive which respects encapsulation paradigm since it would be used throught the pojo public members().

Here's a simple runtime directive marker interface so we can illustrate concept:

// org.martin.metatag.Filter
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface Filter { }

And here we have an implementation class that uses org.apache.commons.beanutils.BeanUtils impl:

import java.io.Serializable;
import java.beans.PropertyDescriptor;
import org.apache.commons.beanutils.BeanUtils;
import org.apache.commons.beanutils.PropertyUtilsBean;
import org.apache.commons.lang.builder.ToStringBuilder;
import org.apache.commons.lang.builder.ToStringStyle;

public class BeanUtil< T extends Serializable > {

 public String toString( T entity ) throws Exception {
   return ToStringBuilder.reflectionToString( entity , ToStringStyle.MULTI_LINE_STYLE );
    }
 
 public void populate( T orig , T dest ) throws Exception {
   BeanUtils.copyProperties( dest , orig );
 }

    // @org.martin.metatag.Filter impl
 public void filter( T orig , T dest ) throws Exception {
   PropertyUtilsBean propertyUtilsBean  = new PropertyUtilsBean();  
          PropertyDescriptor[] origDescriptors = propertyUtilsBean.getPropertyDescriptors(orig);
          for (int i = 0; i < origDescriptors.length; i++) {
            String name = origDescriptors[i].getName();
            if ("class".equals(name)) continue;
            if ( origDescriptors[i].getReadMethod().isAnnotationPresent(  org.martin.metatag.Filter.class ) ) continue;
            if ( origDescriptors[i].getWriteMethod().isAnnotationPresent( org.martin.metatag.Filter.class ) ) continue;
            if ( propertyUtilsBean.isReadable ( orig , name ) &&
              propertyUtilsBean.isWriteable( dest , name ) ) {
                 try {
                    Object value = propertyUtilsBean.getSimpleProperty( orig , name );
                    BeanUtils.copyProperty( dest , name , value );
                 } catch (NoSuchMethodException e) { 
                    // deal with it or ignore
                 }
            }
          }
 }
}

Here's our entity pojo with the @Filter:

public class User implements java.io.Serializable {

   private int id;
   private String name;
   private Set roles;

   public int getId() {
       return id;
   }   
   public void setId(int id) {
       this.id = id;
   }

   public String getName() {
       return name;
   }   
   public void setName(String n){
       name = n
   }

   @org.martin.metatag.Filter
   public java.util.Set getUserRoleses() {
       return roles;
   }   
   public void setUserRoleses(java.util.Set s) {
       roles = s;
   }
}

And finally here's a snippet for util class:

   User orig = ( User ) ormTemplate.get( User.class , id );// fetch Entity from DB
   User dest = new User();                                // allocate a destination TO entity
   (new BeanUtil()).filter( orig , dest );                 // perform filtering though reflexion
   // result dest obj should be populated with everything but filtered roles, ready to be delegated further

The concept here is that @annotations are a powerful new feature from Java language which allow us to control | manipulate object state | behaviour. This illustration depicts how to use a filter mechanism dealing with pojo getters() reflective population which is very common in java apps.

Imagine taking this concept to many different levels, new frameworks are starting to inject more and more metadata directives, this is a PoC impl which worked for me when serializing pojos into AMF binary protocol, and also dealing with hibernate entities which are proxied with CGLIB bytecode generation library such as LazyInitializer, Callback, etc, etc.

Hope you find this useful ;-), let me know your thoughts.

Cheers.

JNLP Experiment

2009-05-06T09:19:00.000-07:00

Hello!

Something triggered me to evaluate at existing technologies to distribute + install rich java desktop clients in contrast of new RIA apps that run on browser + their own runtime (which has to be previously installed on the client), therefore if one already has a Java Swing desktop app, the distribution / install to clients could be simply managed by JavaWebStart [JNLP], so I decided to give it a try and experiment using a host such as AppEngine in static mode to serve the .jnlp file as well as java libraries.

I did not have a GUI app to play with so I decided to provide an existing one such as the Groovy UI console as an example.

Here's a ?.jnlp file for demonstration purposes:



Groovy

Groovy UI Console launched as JNLP.

And it can be run + tested @ here!

You'll have to check 'Always trust content from this publisher' because JNLP specs force you to sign jars ('if unrestricted access to the local system is needed') and I did that with a temp keystore/cert. Note: as you can imagine this opens a security risk, what you run in that console is at your own risk ;-) this is just to illustrate a concept. And perhaps you'll have to double-click on downloaded jnlp file due to MIME application/x-java-jnlp-file not being understood, just in case you may need to instruct your browser to open and associate it with JAVA_HOME/bin/javaws, part of this could be taken care of (server side) by a servlet wrapper which dynamically writes appropriate mime content-type header and contents to result output stream.

Once it downloads and runs, you should be able to see the groovy gui console there at your fingertips to execute some new dynamic language stuff ;-)

Update 05/10/09: I uploaded a simpler sample which downloads much faster since jar is only 19k and can be tested here. It's a simple PoC swing ui utility which builds from user.dir a tree component so you can easily navigate to or open resources for 'windows' OS, make sure you click on desired tree nodes so you can delegate to windows file protocol handler, here's the trick for that:

public static void openResourceInWindows( String resourcePahName ) throws Exception {
Runtime.getRuntime().
exec("rundll32 url.dll,FileProtocolHandler " + resourcePahName );   
}

Hope you find this interesting!

Cheers!

AppEngine & Adobe BlazeDS (fix)

2009-04-25T09:36:00.001-07:00

http://riaspringblaze.appspot.com https://riaspringblaze.appspot.com

Hello, Today I was able to get [ 'remoting only' ] BlazeDS (Community release 3.2.0.3978) to work on GAE, there seems to be an issue with current production GAE Servlet Container in which consequently causes BlazeDS class flex.messaging.io.amf.AbstractAmfInput.java when gets InputStream http request to throw an EOFException = (GAE 1339) resulting a context to resolve into MessageIOConstants.STATUS_NOTAMF in the flex.messaging.endpoints.amf.SerializationFilter class.

Here's my un-official 'fix' to the issue (which just worked for me this morning): So couple things: [1] First make sure config file services-config.xml has

false

, so it's not JMX managed (restricted by GAE)


false

false

[2] And Second: on BlazeDS source look for flex.messaging.io.amf.AbstractAmfInput.java and replace line #58 (aprox) this.in = new DataInputStream( in ); for this.in = new DataInputStream( new java.io.BufferedInputStream( in ) ); So it would look as:

public abstract class AbstractAmfInput extends AmfIO implements ActionMessageInput
{
protected BeanProxy beanProxy = new BeanProxy();

protected DataInputStream in = null;

public AbstractAmfInput(SerializationContext context)
{
super(context);
}

public void setInputStream(InputStream in)
{
this.in = new DataInputStream( new BufferedInputStream( in ) );
}

... more

run BlazeDS ant build.xml in the following targets order : clean,sdk,common,core and replace the flex-messaging-core.jar into your application WEB-INF/lib folder, and deploy new war to GAE cloud.

The one I deployed also contains Spring (3.0.0.M2) and http://www.springsource.org/spring-flex(1.0.0.RC1)

Here's a demo app to demonstrate that it works:

(Note: Flex app is a proof of concept for demonstration purpose)

http://riaspringblaze.appspot.com https://riaspringblaze.appspot.com For those of you interested in compiled flex-messaging-core.jar drop a request in this blog's comments and I'll send it asap (do not send to my email directly, thanks). Note: this is an unofficial community effort, so it is delivered 'as is' for you to experiment with ;-) Update on 4/28/09, Another issue dealing with server sessions on messaging flex.messaging.endpoints.BaseHTTPEndpoint class detects a REQUEST_ATTR_DUPLICATE_SESSION_FLAG and throws "flex.messaging.request.DuplicateSessionDetected", it'd be nice if one could add to config file a no session management parameter. So for now on [line# 405 aprox] flex.messaging.endpoints.BaseHTTPEndpoint class I basically removed the duplicateSessionDetected check (temporarily) till this matures a bit more.

Which by the way on deployment [appengine-web.xml] descriptor you'll need:

true

Update on 09/28/2009: http://bugs.adobe.com/jira/browse/BLZ-444 I've received lots of emails asking me what libraries I'm packing under /WEB-INF/lib from G-sdk along with all other 3rd party dependencies, here's the list I have in my war (you may need more or less depending on what your app does):

martin@martin-ubuntu9 ~/projs/blazeappengine/war/WEB-INF/lib
$ ls -1
antlr-3.0.1.jar
aopalliance.jar
appengine-api-1.0-sdk-1.2.0.jar
asm-3.1.jar
aspectjrt.jar
backport-util-concurrent.jar
cfgatewayadapter.jar
cglib-2.2.jar
commons-beanutils.jar
commons-collections-3.1.jar
commons-fileupload-1.2.1.jar
commons-lang.jar
commons-logging-1.1.jar
concurrent.jar
datanucleus-appengine-1.0.0.final.jar
datanucleus-core-1.1.0.jar
datanucleus-jpa-1.1.0.jar
flex-messaging-common.jar
flex-messaging-core.jar
flex-messaging-opt.jar
flex-messaging-proxy.jar
flex-messaging-remoting.jar
geronimo-jpa_3.0_spec-1.1.1.jar
geronimo-jta_1.1_spec-1.1.1.jar
jackson-core-asl-0.9.9-6.jar
jdo2-api-2.3-SNAPSHOT.jar
org.springframework.aop-3.0.0.M2.jar
org.springframework.aspects-3.0.0.M2.jar
org.springframework.beans-3.0.0.M2.jar
org.springframework.context-3.0.0.M2.jar
org.springframework.context.support-3.0.0.M2.jar
org.springframework.core-3.0.0.M2.jar
org.springframework.expression-3.0.0.M2.jar
org.springframework.flex-1.0.0.RC1.jar
org.springframework.transaction-3.0.0.M2.jar
org.springframework.web-3.0.0.M2.jar
org.springframework.web.servlet-3.0.0.M2.jar
xalan.jar

Let me know how it goes.....

Also, I'd appreciate comments/feed-back/critiques on my blog as well so we can all share experiences / discoveries, thanks. Cheers!

Simple Java CP Builder & Bootstrap

2009-04-23T13:09:00.000-07:00

Greetings! How many time have you had the need to launch your java app and you wish you could organize a /lib folder such that it would look like this with neat sub-folders with their pertaining jars:

One could then write a batch launcher such as:

set ROOT=.
set CP=%ROOT%/lib/com.app.desktop_1_0_0.jar
set CP=%CP%;%ROOT%/lib/antlr/antlr-2.7.6.jar
set CP=%CP%;%ROOT%/lib/antlr/antlr-runtime-3.1.1.jar
set CP=%CP%;%ROOT%/lib/commons/commons-beanutils.jar
set CP=%CP%;%ROOT%/lib/commons/commons-codec-1.3.jar
set CP=%CP%;%ROOT%/lib/other/x.jar
set CP=%CP%;%ROOT%/lib/other/y.jar
..and more here!!
set JAVA_EXE=%JAVA_HOME%\bin\java
set JVM_OPTS=-ea:com.martin... -Xms128m -Xmx256m -XX:PermSize=128m -XX:MaxPermSize=256m
set CLASS=com.martin.server.Bootstrap
set CONFIG=config/log4j.xml config/config.xml
"%JAVA_EXE%" -cp %CP% %JVM_OPTS% %CLASS% %CONFIG%

Here's an approach to let a java class build dynamically a CP from a root folder down recursively for us and could easily port and run on different environments (note: this is a guideline, below code is incomplete):

// recurseFileSystem from root folder
MyClassPath classpath = new MyClassPath( root );
String cp = System.getProperty("java.class.path");
System.setProperty("java.class.path", cp + ';' + classpath.getCPListAsString() );
Thread.currentThread().setContextClassLoader( classpath.getClassLoader() );
Policy policy = Policy.getPolicy();
if (policy != null)
policy.refresh();
classpath.getClassLoader().loadClass(classname).
getDeclaredMethod("main",paramTypes).invoke(null,params);

Where the trick is performing a file system recursive jar fetcher and inject it into System.setProperty("java.class.path",x); and then invoking a main from a arg class specified, below is a sample recursive file system code:

private void recurseFileSystem( File dir ) throws IOException {

File[] jars = dir.listFiles( new FilenameFilter()  {
public boolean accept(File dir, String name)  {
 String namelc = name.toLowerCase();
   return namelc.endsWith(".jar");
 // could add more file extensions here
}
} );

for (int i=0; jars!=null && icp+=":"  !#")+jar);
}

File[] dirs = dir.listFiles(
new FilenameFilter()  {
   public boolean accept(File dir, String name)  {
   return dir.isDirectory();
   }
     }
);

for (int i=0;i

By doing so here's the result from a console with our CP built for us:

Root  -> C:/martin/projs/demo
Config-> C:/martin/projs/demo/config
Class -> com.martin.server.Bootstrap
Libs  -> C:/martin/projs/com.martin.svr/lib
Args  -> log4j.xml config.xml
->cp+=C:/martin/projs/demo/lib/com.martin.bootstrap_1_0_0.jar
->cp+=C:/martin/projs/demo/lib/com.martin.svr_1_0_0.jar
->cp+=C:/martin/projs/demo/lib/antldr/antlr-2.7.6.jar
->cp+=C:/martin/projs/demo/lib/antldr/antlr-runtime-3.1.1.jar
->cp+=C:/martin/projs/demo/lib/commons/commons-beanutils.jar
->cp+=C:/martin/projs/demo/lib/commons/commons-codec-1.3.jar
->cp+=C:/martin/projs/demo/lib/commons/commons-lang.jar
->cp+=C:/martin/projs/demo/lib/commons/commons-logging-1.1.jar
->cp+=C:/martin/projs/demo/lib/jetty/jetty-naming-6.1.16.jar
->cp+=C:/martin/projs/demo/lib/jetty/jetty-util-6.1.16.jar
->cp+=C:/martin/projs/demo/lib/jetty/jetty.jar
->cp+=C:/martin/projs/demo/lib/jgroup/jgroups-core.jar
context            ->C:/martin/projs/com.martin.svr
classloader        ->java.net.URLClassLoader
classloader.parent ->sun.misc.Launcher$AppClassLoader
================[  start  ]=============

Console example:

So we would invoke our app simply as: java -jar run.jar ./ /lib org.myapp.MyClassWithMain

//usage : java -jar run.jar rootPath libPathFromRoot className cmdArguments Cheers! Martin

Lean Server Container

2009-04-23T09:51:00.000-07:00

Greetings!

I have worked in the past with full stack J2EE: JBoss,Websphere, and servlet containers: Tomcat, Jetty, Resin, although sometimes I felt some were too heavy with many bells + whistles and plain servlet container lacked few other features, I was thinking perhaps I could glue together a middle ground container assembled via a dependency injection framework.

The result was that I ended up assembling my own simple container as an experiment, here's the full stack so far:

[0] Log4j for logging.

[1] Spring for main configuration and IoC.

[2] JMX Kernel for managed MBeans support.

[3] Embbeding Jetty servlet container.

[4] JGroups for a simple cloud support.

[5] Embbeded Hypersonic RDBM in-proc DB support.

[6] Hibernate for ORM support.

[7] A simple FIFO Queue support.

[8] A simple Map mem cache with fail over capabilities via JGroups.

And many other collaboration frameworks such as Jasper reporting engine, JFreeCharting, Drools rules engine, Quartz scheduling, BlazeDS for AMF serializations, etc, etc.

Container bootstraps in less that 5 seconds with all features, here's the start up sequence console:

I'd like to share my spring main config file to depict all collaboration objects:



     Martin Z. 04/23/2009
     Main configuration.
     Register in bootstrap.services class all [Services]





    
    
        
          cluster
          dbserver          
  ormservice 
  mqservice
  cacheservice
  http

Now I prototype quickly on my self-managed container, I have tested portability by deploying a war web app on JBoss 5 with [ -cweb ] configuration and it all worked just fine.

Anyways, this is it for today ;-)

Cheers!

Martin

AppEngine 2nd Experiment

2009-04-22T12:11:00.000-07:00

http://riaconcept.appspot.com/ria/Wind.swf

https://riaconcept.appspot.com/ria/Wind.swf

So I added cron support for a simple scheduled job that fetches (via fetchurl impl wrapped as java.net.URL) weather data from sources and places in a local cache. Also I played with yahoo map swc component.

cron.xml


/urltoaccess
Fetch the weather cache every 15 minutes
every 15 minutes of 09:00 18:00
America/Tijuana

Although schedule for time range is still not implemented @ GAE, which I added a request here .

Google AppEngine - RIA PoC Experiment

2009-04-15T07:15:00.000-07:00

So I decided to take Google’s AppEngine for a quick test drive last night, here’s the result after few deploys :

(Note: not all features are implemented, this is a quick proof of concept ;-)

http://riaconcept.appspot.com

https://riaconcept.appspot.com

This is a Flex simple app based on (http://www.thinserverarchitecture.com) principle running on Google's AppEngine + cloud:

Experiment Results:

[1] Careful using 3rd party libraries that use reflexion, even if library introspects your own classes, Google sandboxes it, application still deploys without warning but fails on runtime @ cloud, even though app works fine on local development environment with current Google’s SDK 1.2.0 ver. Oh and spawning threads are a no no also.

Here’s link so you can check what JRE classes are available:

http://code.google.com/appengine/docs/java/jrewhitelist.html

[2] Instead of doing JSP on view I decided to do the UI as RIA via Flex which talks over HTTP / XML to a servlet, I'' try the BlazeDS / AMF soon.

[3] Make sure you specify static + non-static content @ \WEB-INF\appengine-web.xml deployment descriptor and if you want SSL support.

[4] JDO POJO Entities via annotation demarcation is nice + easy, example:

@PersistenceCapable(identityType = IdentityType.APPLICATION)

public class User {

@PrimaryKey

@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)private Long id;

@Persistent private String name;

@Persistent private String email;

@Persistent private Date created;

public Long getId() { return id; }

…setters() and getters()

So dealing with Entity persistant was almost a snap, I just wish JDOQL was a bit more robust, but overall if you know any ORM you would feel very familiar to persist and retrieve entities from Bigtable thru JDO wrapper (datanuleous).

[5] My PDF proxy does not work, not sure why yet, but perhaps it has to do with http header Content-Disposition",method + ";filename=" + name

[6] Internal dev SDK servlet container is Jetty with jasper support for JSP, and they provide ANT scripts for those who do not want to use Eclipse plug-in.

[7] AppEngine web console is easy to navigate and intuitive, you can see logs and filter them by different levels, and details on resource consumption is well displayed.

current free quotas

request size - 10 megabytes
response size - 10 megabytes
request duration - 30 seconds
simultaneous dynamic requests - 30
maximum number of application files - 1,000
maximum number of static files - 1,000
maximum size of an application file - 10 megabytes
maximum size of a static file - 10 megabytes
maximum total size of all application and static files - 150 megabytes

Overall, for couple hrs of play, I experienced after few deploys the google cloud, you have to abide to few rules + (expected) restrictions here and there but it’s still on a early stage. With free quota one can only experiment to a certain extent but gives you enough to get your feet wet.

Cheers. ;-)

Have fun!

Martin