Sunday, January 10, 2010

Wicket on Google App Engine

As a continuation of the proof of concept application I started looking at what UI framework I could use. JSF/Richfaces was my preferred UI layer as I have a fair bit of experience with it but after a few days working around a number issues to get it working I reached a final road block and came to the conclusion it just can't work on app engine at this stage. There are too many issues with security permissions and non white listed classes when trying to run it on app engine. An alternative framework had to be found. Wicket peaked my interest do to its component based programmatic approach and that others had already demonstrated some success using it on app engine.

I found a blog by Nick Wiedenbrueck here which really got me started.

Limitations I've Encountered




  • Still some problems with security permissions and app engine white lists

  • Some performance problems




Step 1. Add necessary dependencies to maven project.



First thing is to add the necessary dependencies to your maven project. This should pull in everything you need.





<dependency>
<groupId>org.apache.wicket</groupId>
<artifactId>wicket</artifactId>
<version>1.4.3</version>
</dependency>
<dependency>
<groupId>org.apache.wicket</groupId>
<artifactId>wicket-spring</artifactId>
<version>1.4.3</version>
</dependency>
<dependency>
<groupId>org.apache.wicket</groupId>
<artifactId>wicket-extensions</artifactId>
<version>1.4.3</version>
</dependency>
<dependency>
<groupId>org.apache.wicket</groupId>
<artifactId>wicket-auth-roles</artifactId>
<version>1.4.3</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.15</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.4.2</version>
</dependency>


Step 2. Configure Web.xml



All you need to do to add wicket to your web application is to add the wicket filter to your web.xml and point it to your own Wicket Application class.


<filter>
<filter-name>WicketFilter</filter-name>
<filter-class>org.apache.wicket.protocol.http.WicketFilter</filter-class>
<init-param>
<param-name>applicationClassName</param-name>
<param-value>com.agilewombat.blog.MyApplication</param-value>
</init-param>
</filter>

<filter-mapping>
<filter-name>WicketFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>


Step 3. Create Wicket Application Class



Your wicket application class will look like a standard wicket application with one main exception. The GAE restricts access to classes in the file io classes, so wickets second level session store can't work and must be disabled (line 10). Also the app engine prevents spawning of new threads so the standard resource modification watcher must be either disable or overridden with a version that doesn't spawn threads (line 17). Additionally the class below adds spring integration and enables wicket ajax features.


public class MyApplication extends AuthenticatedWebApplication {
@Override
public Class<? extends Page> getHomePage() {
return HomePage.class;
}

@Override
protected ISessionStore newSessionStore() {
// disabled second level cache because it relies on File.io classes (incompatible with GAE)
return new HttpSessionStore(this);
}

@Override
protected void init() {
super.init();
addComponentInstantiationListener(new SpringComponentInjector(this));
getResourceSettings().setResourceWatcher(new GaeModificationWatcher());
enableAjaxFeatures();
}

@Override
public RequestCycle newRequestCycle(final Request request, final Response response) {
return new MyWebRequestCycle(this, (WebRequest) request, response);
}

private void enableAjaxFeatures() {
getResourceSettings().setThrowExceptionOnMissingResource(false);
getRequestCycleSettings().addResponseFilter(new AjaxServerAndClientTimeFilter());
getDebugSettings().setAjaxDebugModeEnabled(true);

}
}


Step 4. Create GaeModificationWatcher


The following is what I wrote as an alternative modification watcher to the built in wicket one (so development use only and use at own risk). This one doesn't use threads but will use the poll interval to limit the number of times notifications can be sent.


public class GaeModificationWatcher implements IModificationWatcher {
private static final Logger LOG = LoggerFactory.getLogger(GaeModificationWatcher.class);

ConcurrentHashMap<IModifiable, Set<IChangeListener>> listenersMap = new ConcurrentHashMap<IModifiable, Set<IChangeListener>>();

Duration pollFrequency;
Time lastCheckTime;
Object timeCheckLock = new Object();

@Override
public boolean add(IModifiable modifiable, IChangeListener listener) {
checkResources();
Set<IChangeListener> listeners = listenersMap.putIfAbsent(modifiable, new HashSet<IChangeListener>());
return listeners.add(listener);
}

@Override
public void destroy() {
// do nothing

}

@Override
public Set<IModifiable> getEntries() {
return listenersMap.keySet();
}

@Override
public IModifiable remove(IModifiable modifiable) {
if (listenersMap.remove(modifiable) != null) {
return modifiable;
} else {
return null;
}
}

@Override
public void start(Duration pollFrequency) {
LOG.debug("Starting watcher");
synchronized (timeCheckLock) {
lastCheckTime = Time.now();
this.pollFrequency = pollFrequency;
}
}

public void checkResources() {
Time now = Time.now();

Time timeCheck;
synchronized (timeCheckLock) {
if (lastCheckTime == null) {
return; // not started
}

Time nextTimeCheck = lastCheckTime.add(pollFrequency);
if (nextTimeCheck.after(now)) {
return; // nothing to do, not ready
}

// lets go
timeCheck = this.lastCheckTime;
this.lastCheckTime = now;
}

Set<Entry<IModifiable, Set<IChangeListener>>> entrySet =
new HashSet<Entry<IModifiable, Set<IChangeListener>>>(listenersMap.entrySet());

for (Entry<IModifiable, Set<IChangeListener>> entry : entrySet) {
if (timeCheck.before(entry.getKey().lastModifiedTime())) {
LOG.debug("Found modification, notifying listeners of change");
for (IChangeListener listener : entry.getValue()) {
listener.onChange();
}
}
}
}

}


Step 5. Create MyWebRequestCycle


The MyWebRequestCycle is a custom WebRequestCycle that just ensures that the modification watcher is given a chance to check every request (because we don't have a seperate thread to check.



class MyWebRequestCycle extends WebRequestCycle {

MyWebRequestCycle(final WebApplication application,
final WebRequest request, final Response response) {
super(application, request, response);
}

@Override
protected void onBeginRequest() {
if (getApplication().getConfigurationType().equals(Application.DEVELOPMENT)) {
final GaeModificationWatcher resourceWatcher = (GaeModificationWatcher) getApplication()
.getResourceSettings().getResourceWatcher(true);
resourceWatcher.checkResources();
}

}
}


Step 6. Create Home Page


Finally finish off with the actual Home Page. This needs a Java file and Html file in the same package.


public class HomePage extends WebPage {
public HomePage() {
add(new Label("label", new Model("Hello, World")));
}
}



<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
xmlns:wicket="http://wicket.apache.org/dtds.data/wicket-xhtml1.4-strict.dtd">
<head>
<title>Home Page</title>
</head>
<body>
<span wicket:id="label"></span>
</body>
</html>



Thats it!

Monday, December 7, 2009

Spring, JDO and Google App Engine

For a proof of concept of app engine, I've manage to put together a simple app using spring, jdo and gae. There are a few small gotchas along the way but as long as you keep it simple most things appear to work.
I chose JDO because the GAE persistence layer is built using datanucleus which uses JDO as its core api. JPA is implemented by datanucleus as an extension but my initial tests gave me a lot of problems that were hard to resolve due to the uninformative error messages.
A note, I found datanucleus error messages about mistakes in defining relationships etc. were extremely poor compared to hibernate and I hope this is an area they improve in.

Limitations



  • GAE doesn't support JNDI and spring's component scan autowiring tries to use it

  • Limitations on JDO implementation including no inheritence, bad support for un-owned relationships and only objects in the same group can be saved in one transactions



Step 1. Add necessary dependencies to maven project.


For a maven project this is simply a matter of declaring the dependencies in your pom.


<!--app engine related dependencies-->
<dependency>
<groupid>com.google.appengine</groupid>
<artifactid>appengine-api-1.0-sdk</artifactid>
<version>${appengine.version}</version>
</dependency>
<dependency>
<groupid>com.google.appengine.orm</groupid>
<artifactid>datanucleus-appengine</artifactid>
<version>${datanucleus-appengine.version}</version>
</dependency>
<dependency>
<groupid>javax.jdo</groupid>
<artifactid>jdo2-api</artifactid>
<version>${jdo2-api.version}</version>
</dependency>
<dependency>
<groupid>org.datanucleus</groupid>
<artifactid>datanucleus-core</artifactid>
<version>${datanucleus.version}</version>
</dependency>
<dependency>
<groupid>org.datanucleus</groupid>
<artifactid>datanucleus-jpa</artifactid>
<version>1.1.5</version>
</dependency>
<!--jstl -->
<dependency>
<groupid>javax.servlet</groupid>
<artifactid>jstl</artifactid>
<version>1.1.2</version>
</dependency>
<dependency>
<groupid>taglibs</groupid>
<artifactid>standard</artifactid>
<version>1.1.2</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-core</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-web</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-beans</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-orm</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-aop</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-jdbc</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>cglib</groupid>
<artifactid>cglib</artifactid>
<version>2.2</version>
</dependency>

The data nucleus dependencies aren't in the central repo so you should add the following repo to you're pom

<pluginRepository>
<id>datanucleus</id>
<url>http://www.datanucleus.org/downloads/maven2</url>
</pluginRepository>


Step 2. Configure Spring Application Context


Setting up spring is quite simple, the only issue that needs a work around is that the annotation config used by spring to automagically inject depends on JNDI even if you're not using it. This causes as ClassNotFoundException to be thrown when deploying to the live GAE.
The simple work around for this is to declare internalPersistenceAnnotationProcessor bean in your config which tricks spring into thinking this class is already loaded and doesn't attempt to load it. I'm not totally happy with this work around but it works for the moment.

Create an applicationContext.xml in the class path as below.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
xmlns:tx="http://www.springframework.org/schema/tx" xmlns:context="http://www.springframework.org/schema/context"
xmlns:util="http://www.springframework.org/schema/util" xmlns:p="http://www.springframework.org/schema/p"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.5.xsd">

<!-- workaround appengine lack of JNDI support -->
<bean id="org.springframework.context.annotation.internalPersistenceAnnotationProcessor"
class="java.lang.String" />
<!-- End workaround -->

<context:component-scan base-package="com.agilewombat"/>
<context:annotation-config />

<tx:annotation-driven />

<!-- JDO Specific -->
<bean id="persistenceManagerFactory"
class="org.springframework.orm.jdo.LocalPersistenceManagerFactoryBean">
<property name="persistenceManagerFactoryName" value="transactions-optional" />
</bean>

<bean id="transactionManager" class="org.springframework.orm.jdo.JdoTransactionManager">
<property name="persistenceManagerFactory" ref="persistenceManagerFactory" />
</bean>
<!-- End JDO Specific -->
</beans>


Add spring to your web.xml


<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>classpath:applicationContext*.xml</param-value>
</context-param>

<!-- spring context loader lister -->
<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>



Step 3. Configure JDO



This is a standard JDO config. Create a jdoconfig.xml file in the META-INF directory of your resources.


<?xml version="1.0" encoding="utf-8"?>
<jdoconfig xmlns="http://java.sun.com/xml/ns/jdo/jdoconfig"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://java.sun.com/xml/ns/jdo/jdoconfig">

<persistence-manager-factory name="transactions-optional">
<property name="javax.jdo.PersistenceManagerFactoryClass"
value="org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory"/>
<property name="javax.jdo.option.ConnectionURL" value="appengine"/>
<property name="javax.jdo.option.NontransactionalRead" value="true"/>
<property name="javax.jdo.option.NontransactionalWrite" value="true"/>
<property name="javax.jdo.option.RetainValues" value="true"/>
<property name="datanucleus.appengine.autoCreateDatastoreTxns" value="true"/>
</persistence-manager-factory>
</jdoconfig>


Step 4. Generic Dao


You can write a dao that can be use for any of your domain objects by wrapping the spring JDO Template. This will also help you if you decide to switch to JPA. One major limitation of GAE is that transactions can only work on objects in the same group. This is typically will mean one object and its children. As such its not very usefuly to have transactional boundaries on the service layer as typical because you can't do much more than dao operations in a transaction. Instead transactions can be placed on the dao.


@Repository
@Transactional(readOnly = false)
public class GenericDao {

@Autowired
private PersistenceManagerFactory pmf;

private JdoTemplate jdoTemplate;

@PostConstruct
void createJdoTemplate() {
jdoTemplate = new JdoTemplate(pmf);
}

@Transactional(readOnly = true)
public <T extends DomainObject> T findByKey(Class<T> clazz, Key id) {
@SuppressWarnings("unchecked")
T entity = (T) jdoTemplate.getObjectById(clazz, id);

if (entity == null) {
throw new ObjectRetrievalFailureException(clazz, id);
}
return entity;
}

@Transactional(readOnly = true)
@SuppressWarnings("unchecked")
public <T extends DomainObject> Collection<T> findAll(Class<T> clazz) {
return jdoTemplate.detachCopyAll(jdoTemplate.find(clazz));
}

@Transactional
@SuppressWarnings("unchecked")
public <T extends DomainObject> void remove(T domainObj) {
T domainObject = (T) jdoTemplate.getObjectById(domainObj.getClass(),
domainObj.getKey());
jdoTemplate.deletePersistent(domainObject);
}

@Transactional(readOnly = true)
@SuppressWarnings("unchecked")
public <T extends DomainObject> Collection<T> findByNamedQuery(Class<T> clazz, String namedQuery, Map<String, Object> values) {
return jdoTemplate.findByNamedQuery(clazz, namedQuery, values);
}


@Transactional
@SuppressWarnings("unchecked")
public <T extends Object> T save(T domainObj) {
return (T) jdoTemplate.makePersistent(domainObj);
}

@Transactional
public void flush() {
jdoTemplate.flush();
}

}

Step 5. Ready To Go - Write Some Domain Objects


Now this is where the fun starts. The following annoying you should be aware of for those coming from hibernate.

  • Inheritence plain doesn't work so avoid it (This is a known bug in the gae layer that should be fixed in next release)

  • Un-owned relationships are not supported. This means to hold a reference to an object that is not a child you can only keep the primary key of the object and must manually dereference it in code.



An example User Class


@PersistenceCapable(identityType = IdentityType.APPLICATION, detachable = "true")
@Queries({
@Query(name = "findByUsername", value = "SELECT FROM User u where u.username == :username")
})
public class User {
private static final long serialVersionUID = 1L;

@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;

@Persistent
private String username;

@Persistent
private Date createdDate;

@Persistent
private Date modifiedDate;

@Persistent(defaultFetchGroup = "true")
private Key createdUser;

@Persistent(defaultFetchGroup = "true")
private Key modifiedUser;

@Persistent
private String password;

@Persistent
private Role role;

...
}


Step 6. Testing



Spring tests can be used as normal, the only thing extra that is required is to setup a base GAE environment. You can extend the following BaseTestFixture in your tests.


public class BaseTestFixture {

private boolean cleanEnvironment = true;

@Before
public void setUp() {
if (cleanEnvironment) {
ApiProxyLocalImpl proxy = new ApiProxyLocalImpl(new File(".")) {};
proxy.setProperty(LocalDatastoreService.NO_STORAGE_PROPERTY, Boolean.TRUE.toString());
ApiProxy.setDelegate(proxy);
ApiProxy.setEnvironmentForCurrentThread(new GoogleAppEngineTestEnvironment());
}
}

@After
public void tearDown() {
if (cleanEnvironment) {
ApiProxy.clearEnvironmentForCurrentThread();
}
}
}

Sunday, December 6, 2009

Code Syntax Highlighting

Any good code blog needs syntax highlighting. Unfortunately, blogger doesn't seem to offer this out of the box. Fortunately, there is a great little javascript package called SyntaxHighlighter written by Alex Gorbatchev here. Hosting for the syntax highlighter javascript is provided on his site so installing it for use in blogger is simply done by editing your blog template's html and inserting the javascript declarations below within the head element.


<link href='http://alexgorbatchev.com/pub/sh/2.1.364/styles/shCore.css' rel='stylesheet' type='text/css'/>
<link href='http://alexgorbatchev.com/pub/sh/2.1.364/styles/shThemeDefault.css' rel='stylesheet' type='text/css'/>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shCore.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushCpp.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushCSharp.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushCss.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushJava.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushJScript.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushPhp.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushPython.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushRuby.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushSql.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushVb.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushXml.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushPerl.js' type='text/javascript'></script>
<script language='javascript'>
SyntaxHighlighter.config.bloggerMode = true;
SyntaxHighlighter.config.clipboardSwf = 'http://alexgorbatchev.com/pub/sh/2.1.364/scripts/clipboard.swf';
SyntaxHighlighter.all();
</script>



You can then wrap your code snippet in a <pre> tag in your code but this isn't perfect because html characters need to be escaped.


<pre class="brush: html">

</pre>



As an alternative you are meant to be able to can use the CDATA method below where html characters don't need to be escaped but it doesn't work for me as blogger doesn't let me post it.


<script class="brush: html" type="syntaxhighlighter"><![CDATA[...]]></script>

Saturday, December 5, 2009

First Post!

Lately I've been working on a couple of projects to experiment with different technologies I don't normally have much use for in my day job (e.g. GAE). As with anything new I've hit a few problems, some just noob mistakes, some limitations and some straight out bugs. In a few instances I've been really frustrated by the lack of information I could find googling and I've ended up with a lot of little bits of information rattling around in my head. So I decided to start this blog to help collect my thoughts on what worked/didn't work and hopefully help others who hit similar problems.