JVM Agent - Class Spy Utility

I've been experimenting recently on a PoC java agent utility which dumps high & low level details for any class loaded by JVM at any point of time.
It does not perform any bytecode instrumentation manipulation / alteration but rather loads all classes available / loaded by the JVM in context into a UI in which it organizes classes by package namespace into a hierarchical tree structure.
This utility comes handy when you need to introspect class bytecode blueprint signatures that may be in question at runtime, similar to the javap utility that ships with jdk.

According to the specs the agent jar must contain:

MANIFEST.MF
...
Agent-Class: org.martin.agent.jvm.JvmAgent
Premain-Class: org.martin.agent.jvm.JvmAgent
Boot-Class-Path: jvmagent_1_0_0.jar
Can-Redefine-Classes: false
Can-Retransform-Classes: false

(note that redefinitions + transformations are not set since neither is needed here)

and method premain (similar to a 'main') signature hook is as follows:
org.martin.agent.jvm.JvmAgent
public static void premain(String agentArgs, Instrumentation inst) { ... }

once we have instrumentation access we can then:
[1] register a custom class transformer

instrumentation.addTransformer( new MyClassLoadTransformer() );
...
class MyClassLoadTransformer implements ClassFileTransformer {
public byte[] transform(
ClassLoader loader,
String fullyQualifiedClassName,
Class classBeingRedefined,
ProtectionDomain protectionDomain,
byte[] classofileBuffer) throws IllegalClassFormatException {
// illustration here!
}
}

[2] and invoke the instrumentation method:
final Class[] clsArr = instrumentation.getAllLoadedClasses();
which feeds the UI by representing pkgs hierarchicaly in a tree:

Once a desired class node is selected, it prompts for high / low level detail?, either selection will dump its details on to System.out, below is a console sample of output:

Instructions:
[1] download jar (only 22+k) and place it either in /lib or /bin of your app
[2] Then you just need to add -javaagent parameter such as:
java -javaagent:jvmagent_1_0_0.jar ...(more options here)
(depending on where it is : relative path -javaagent:../lib/jvmagent_1_0_0.jar , so configure accordingly!)

Agent will launch and load a UI with an initial draw, if your bootstrap process takes a bit longer then at any point you can press refresh button to re-load jvm state, the jvm + your application will be dynamically loading on demand classes based on the dependencies needed, so refresh accordingly.

Bytecode class dump disassembler credits.

Conclusion
Now imagine taking this concept further, profilers per say, or by using bytecode manipulation libraries such as asm or javassist an example is javarebel which allows reloading classes dynamically at runtime (even if making significant changes) without needing to restart container. I do have a working version agent where does hotswap (as long as new bytecode respects same signature and only changes within method implementation), however JVM can become very stale if you violate that limitation (at least up until now in jdk 6+).

Till next time, cheers!

Java shell

I've been having fun writting a utility java shell, I use cygwin on windows but I needed something more integrated with java such has building class paths, invoking java/ant/groovy /os scripts, scanning in context class loader certain classes | @Annotations, etc, etc. And also when I switch to Linux there it is for me as well. Some basic commands are os file system handling such as ls / rm / mk / grep / etc.

Internals use java.io.Console and a custom annotation:

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface Command {
String name()        default "";
String description() default "";
String abbrev()      default "";
String def()         default "";
}

So anytime I need a new command I can annotate it like below and instantly it becomes available via method reflection invocation:

@Command(name="cmd", description="sample cmd", abbrev=".", def="?")
public void cmd( String str ) throws Exception {
...
}

Here's an example below for few common file system cmds: ls , lsr (just like ls but with recursive capabilities building a tree)

Note that commands can be chained 'piped' taking res output as input like in *nix such as:

lsr src | grep '06-11'

And also io redirect '>' result:

ls . | lsr src | grep '06-11' > result.log

Or io redirect appended '>>'

ls . | lsr src | grep '06-11' >> result.log
etc
I'll have to admit that grep command which takes regular expressions to search is my favorite command of them all (specially in windows cuz u know y), it's so powerfull when combined with others.

Here's a list of the help for all cmds implemented so far (keeps growing by the day)

Today I included a disassembler via 'javap' which scans in context class loader via a package prefix and shows public class signatures, it comes handy when you need to introspect bytecode loaded and being currently invoked. It also can open a script file and sequentially execute cmds but for that there's a groovy cmd which I can invoke and run groovy scripts with dyn-language richness, one example is I run [def ant = new AntBuilder()] via groovy ;-).

Now one thing I hate to do is walking by hand on file system typing long paths, so I added a simple 'fs' cmd which launches a swing UI which facilitates file system operations, I hooked most common cmds in a menu and results are directed to console:

I know it looks like a toy but it surely comes handy on day by day con$ole operations, heck I don't know what I'll add next but it sure saves the day, I love being productive with tools that do things my way ;-) and I can easily throw a new feature at it in a matter of minutes.

One final trick (for windows) is that you can invoke it directly from windows file explorer anytime you right-click on a folder (shows on menu):
file: reg.reg

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Directory\shell\jell]
@="jell"
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Directory\shell\jell\command]
@="cmd.exe /k \"cd %L\"&jell"

I'll probably pack it once it becomes stable so I can share as GA, that is if anybody finds this useful,
Well, till next time, cheers...

Ant + Ivy > Repository +

I'd like to share my Ivy configuration that I use to build my Java repository and to manage transitive dependencies / conflicts, first I'd like to point out that I prefer controlling internals, the 'convention over configuration ease' which by the way is a great concept but seems to fall short sooner or later when it comes to opening the hood and morphing things, I like simplicity and once a tool works for you the way you want then leave it as is. I've read many posts regarding Ant <> Maven which at the end of the day the reality is that there's no such a thing as a 'silver bullet' when building complex + modularized projects.
I think that: "it's not just what a tool can do for you magically, but you can do 'with' the tool in a harmonious/flexible/extensible way" ;-)
I guess that's what I love most about Spring framework, BTW both mentioned products are great, evaluate what you're getting into [pros/cons] and choose wisely. My choice = ubiquitous Ant + Ivy cuts it for me.

   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
   xmlns:extra="http://ant.apache.org/ivy/extra"
   xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd"
>
  
   
    This is the main dependency repository list.
   
 
 
    
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
    

Ant (partial) snippet:

Cheers.

AppEngine & BlazeDS (Messaging)

http://riaspringblaze.appspot.com

Hello!

I added a simple messaging test drive to existing flex PoC app, this time I updated Spring version from 2.5.5 to 3.0.0M2 and from flexM1 to flex-1.0.0.RC1 with my already existing 'slight-hacked' core.jar from my previous blog , it went pretty well and here are the results:

Note that Spring changed @ web-application-config.xml to new 'remoting-destination' ->

Here's a snippet flex client set up ->

  


  destination='DESTINATION-TEST' 
  subtopic='TOPIC-TEST'
  channelSet='{channelMsgSet}' 
  message='messageHandler(event)'
  fault='faultMsgHandler(event)'    
  acknowledge='acknowledgeMsgHandler(event)'      
/>

And finally this is my new list of libraries/dependencies deployed to the cloud for your reference->

martin@martin-ubuntu9 ~/projs/blazeappengine/war/WEB-INF/lib
$ ls -1
antlr-3.0.1.jar
aopalliance.jar
appengine-api-1.0-sdk-1.2.0.jar
asm-3.1.jar
aspectjrt.jar
backport-util-concurrent.jar
cfgatewayadapter.jar
cglib-2.2.jar
commons-beanutils.jar
commons-collections-3.1.jar
commons-fileupload-1.2.1.jar
commons-lang.jar
commons-logging-1.1.jar
concurrent.jar
datanucleus-appengine-1.0.0.final.jar
datanucleus-core-1.1.0.jar
datanucleus-jpa-1.1.0.jar
flex-messaging-common.jar
flex-messaging-core.jar
flex-messaging-opt.jar
flex-messaging-proxy.jar
flex-messaging-remoting.jar
geronimo-jpa_3.0_spec-1.1.1.jar
geronimo-jta_1.1_spec-1.1.1.jar
jackson-core-asl-0.9.9-6.jar
jdo2-api-2.3-SNAPSHOT.jar
org.springframework.aop-3.0.0.M2.jar
org.springframework.aspects-3.0.0.M2.jar
org.springframework.beans-3.0.0.M2.jar
org.springframework.context-3.0.0.M2.jar
org.springframework.context.support-3.0.0.M2.jar
org.springframework.core-3.0.0.M2.jar
org.springframework.expression-3.0.0.M2.jar
org.springframework.flex-1.0.0.RC1.jar
org.springframework.transaction-3.0.0.M2.jar
org.springframework.web-3.0.0.M2.jar
org.springframework.web.servlet-3.0.0.M2.jar
xalan.jar

Give it a try, subscribe to pre-determined dest+topic and emulate a msg push from server, you'll see there an asynch msg received by the client consumer, better yet
launch 2 or 3 browsers, subscribe from all and send msgs from only one browser isntance, you'll see all consumers being published since they all subscribe to same
predetermined topic.... ;-)

Cheers.

Bean @Filter Annotation

Hello!

When performing introspections through your own reflective or 3rd party library and specially if your objects are 'proxied' within a certain context, sometimes there's a need to instruct reflective libraries to 'filter' certain class memebers somewhat similar to intrinsic 'transient' keyword (serializations), in this case I thought of a non-intrusive way to annotate with metadata a runtime directive which respects encapsulation paradigm since it would be used throught the pojo public members().

Here's a simple runtime directive marker interface so we can illustrate concept:

// org.martin.metatag.Filter
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface Filter { }

And here we have an implementation class that uses org.apache.commons.beanutils.BeanUtils impl:

import java.io.Serializable;
import java.beans.PropertyDescriptor;
import org.apache.commons.beanutils.BeanUtils;
import org.apache.commons.beanutils.PropertyUtilsBean;
import org.apache.commons.lang.builder.ToStringBuilder;
import org.apache.commons.lang.builder.ToStringStyle;

public class BeanUtil< T extends Serializable > {

 public String toString( T entity ) throws Exception {
   return ToStringBuilder.reflectionToString( entity , ToStringStyle.MULTI_LINE_STYLE );
    }
 
 public void populate( T orig , T dest ) throws Exception {
   BeanUtils.copyProperties( dest , orig );
 }

    // @org.martin.metatag.Filter impl
 public void filter( T orig , T dest ) throws Exception {
   PropertyUtilsBean propertyUtilsBean  = new PropertyUtilsBean();  
          PropertyDescriptor[] origDescriptors = propertyUtilsBean.getPropertyDescriptors(orig);
          for (int i = 0; i < origDescriptors.length; i++) {
            String name = origDescriptors[i].getName();
            if ("class".equals(name)) continue;
            if ( origDescriptors[i].getReadMethod().isAnnotationPresent(  org.martin.metatag.Filter.class ) ) continue;
            if ( origDescriptors[i].getWriteMethod().isAnnotationPresent( org.martin.metatag.Filter.class ) ) continue;
            if ( propertyUtilsBean.isReadable ( orig , name ) &&
              propertyUtilsBean.isWriteable( dest , name ) ) {
                 try {
                    Object value = propertyUtilsBean.getSimpleProperty( orig , name );
                    BeanUtils.copyProperty( dest , name , value );
                 } catch (NoSuchMethodException e) { 
                    // deal with it or ignore
                 }
            }
          }
 }
}

Here's our entity pojo with the @Filter:

public class User implements java.io.Serializable {

   private int id;
   private String name;
   private Set roles;

   public int getId() {
       return id;
   }   
   public void setId(int id) {
       this.id = id;
   }

   public String getName() {
       return name;
   }   
   public void setName(String n){
       name = n
   }

   @org.martin.metatag.Filter
   public java.util.Set getUserRoleses() {
       return roles;
   }   
   public void setUserRoleses(java.util.Set s) {
       roles = s;
   }
}

And finally here's a snippet for util class:

   User orig = ( User ) ormTemplate.get( User.class , id );// fetch Entity from DB
   User dest = new User();                                // allocate a destination TO entity
   (new BeanUtil()).filter( orig , dest );                 // perform filtering though reflexion
   // result dest obj should be populated with everything but filtered roles, ready to be delegated further

The concept here is that @annotations are a powerful new feature from Java language which allow us to control | manipulate object state | behaviour. This illustration depicts how to use a filter mechanism dealing with pojo getters() reflective population which is very common in java apps.

Imagine taking this concept to many different levels, new frameworks are starting to inject more and more metadata directives, this is a PoC impl which worked for me when serializing pojos into AMF binary protocol, and also dealing with hibernate entities which are proxied with CGLIB bytecode generation library such as LazyInitializer, Callback, etc, etc.

Hope you find this useful ;-), let me know your thoughts.

Cheers.

JNLP Experiment

Hello!

Something triggered me to evaluate at existing technologies to distribute + install rich java desktop clients in contrast of new RIA apps that run on browser + their own runtime (which has to be previously installed on the client), therefore if one already has a Java Swing desktop app, the distribution / install to clients could be simply managed by JavaWebStart [JNLP], so I decided to give it a try and experiment using a host such as AppEngine in static mode to serve the .jnlp file as well as java libraries.

I did not have a GUI app to play with so I decided to provide an existing one such as the Groovy UI console as an example.

Here's a ?.jnlp file for demonstration purposes:

Groovy

Groovy UI Console launched as JNLP.

And it can be run + tested @ here!

You'll have to check 'Always trust content from this publisher' because JNLP specs force you to sign jars ('if unrestricted access to the local system is needed') and I did that with a temp keystore/cert. Note: as you can imagine this opens a security risk, what you run in that console is at your own risk ;-) this is just to illustrate a concept.
And perhaps you'll have to double-click on downloaded jnlp file due to MIME application/x-java-jnlp-file not being understood, just in case you may need to instruct your browser to open and associate it with JAVA_HOME/bin/javaws, part of this could be taken care of (server side) by a servlet wrapper which dynamically writes appropriate mime content-type header and contents to result output stream.

Once it downloads and runs, you should be able to see the groovy gui console there at your fingertips to execute some new dynamic language stuff ;-)

Update 05/10/09: I uploaded a simpler sample which downloads much faster since jar is only 19k and can be tested here. It's a simple PoC swing ui utility which builds from user.dir a tree component so you can easily navigate to or open resources for 'windows' OS, make sure you click on desired tree nodes so you can delegate to windows file protocol handler, here's the trick for that:

public static void openResourceInWindows( String resourcePahName ) throws Exception {
Runtime.getRuntime().
exec("rundll32 url.dll,FileProtocolHandler " + resourcePahName );   
}

Hope you find this interesting!

Cheers!

AppEngine & Adobe BlazeDS (fix)

http://riaspringblaze.appspot.com

Hello,

Today I was able to get [ 'remoting only' ] BlazeDS (Community release 3.2.0.3978) to work on GAE, there seems to be an issue with current production GAE Servlet Container in which consequently causes BlazeDS class flex.messaging.io.amf.AbstractAmfInput.java when gets InputStream http request to throw an EOFException = (GAE 1339) resulting a context to resolve into MessageIOConstants.STATUS_NOTAMF in the flex.messaging.endpoints.amf.SerializationFilter class.

Here's my un-official 'fix' to the issue (which just worked for me this morning):

So couple things:

[1] First make sure config file services-config.xml has

false

, so it's not JMX managed (restricted by GAE)

false

false

[2] And Second: on BlazeDS source look for
flex.messaging.io.amf.AbstractAmfInput.java and replace line #58 (aprox)

this.in = new DataInputStream( in );
for
this.in = new DataInputStream( new java.io.BufferedInputStream( in ) );

So it would look as:

public abstract class AbstractAmfInput extends AmfIO implements ActionMessageInput
{
protected BeanProxy beanProxy = new BeanProxy();

protected DataInputStream in = null;

public AbstractAmfInput(SerializationContext context)
{
super(context);
}

public void setInputStream(InputStream in)
{
this.in = new DataInputStream( new BufferedInputStream( in ) );
}

... more

run BlazeDS ant build.xml in the following targets order : clean,sdk,common,core and replace the flex-messaging-core.jar into your application WEB-INF/lib folder, and deploy new war to GAE cloud.

The one I deployed also contains Spring (3.0.0.M2) and http://www.springsource.org/spring-flex(1.0.0.RC1)

Here's a demo app to demonstrate that it works:

(Note: Flex app is a proof of concept for demonstration purpose)

http://riaspringblaze.appspot.com

For those of you interested in compiled flex-messaging-core.jar drop a request in this blog's comments and I'll send it asap (do not send to my email directly, thanks).

Note: this is an unofficial community effort, so it is delivered 'as is' for you to experiment with ;-)

Update on 4/28/09, Another issue dealing with server sessions on messaging flex.messaging.endpoints.BaseHTTPEndpoint class detects a REQUEST_ATTR_DUPLICATE_SESSION_FLAG and throws "flex.messaging.request.DuplicateSessionDetected", it'd be nice if one could add to config file a no session management parameter. So for now on [line# 405 aprox] flex.messaging.endpoints.BaseHTTPEndpoint class I basically removed the duplicateSessionDetected check (temporarily) till this matures a bit more.

Which by the way on deployment [appengine-web.xml] descriptor you'll need:

true

Update on 09/28/2009: http://bugs.adobe.com/jira/browse/BLZ-444

I've received lots of emails asking me what libraries I'm packing under /WEB-INF/lib from G-sdk along with all other 3rd party dependencies, here's the list I have in my war (you may need more or less depending on what your app does):

martin@martin-ubuntu9 ~/projs/blazeappengine/war/WEB-INF/lib
$ ls -1
antlr-3.0.1.jar
aopalliance.jar
appengine-api-1.0-sdk-1.2.0.jar
asm-3.1.jar
aspectjrt.jar
backport-util-concurrent.jar
cfgatewayadapter.jar
cglib-2.2.jar
commons-beanutils.jar
commons-collections-3.1.jar
commons-fileupload-1.2.1.jar
commons-lang.jar
commons-logging-1.1.jar
concurrent.jar
datanucleus-appengine-1.0.0.final.jar
datanucleus-core-1.1.0.jar
datanucleus-jpa-1.1.0.jar
flex-messaging-common.jar
flex-messaging-core.jar
flex-messaging-opt.jar
flex-messaging-proxy.jar
flex-messaging-remoting.jar
geronimo-jpa_3.0_spec-1.1.1.jar
geronimo-jta_1.1_spec-1.1.1.jar
jackson-core-asl-0.9.9-6.jar
jdo2-api-2.3-SNAPSHOT.jar
org.springframework.aop-3.0.0.M2.jar
org.springframework.aspects-3.0.0.M2.jar
org.springframework.beans-3.0.0.M2.jar
org.springframework.context-3.0.0.M2.jar
org.springframework.context.support-3.0.0.M2.jar
org.springframework.core-3.0.0.M2.jar
org.springframework.expression-3.0.0.M2.jar
org.springframework.flex-1.0.0.RC1.jar
org.springframework.transaction-3.0.0.M2.jar
org.springframework.web-3.0.0.M2.jar
org.springframework.web.servlet-3.0.0.M2.jar
xalan.jar

Let me know how it goes.....

Also, I'd appreciate comments/feed-back/critiques on my blog as well so we can all share experiences / discoveries, thanks.
Cheers!