Monday, December 7, 2009

Spring, JDO and Google App Engine

For a proof of concept of app engine, I've manage to put together a simple app using spring, jdo and gae. There are a few small gotchas along the way but as long as you keep it simple most things appear to work.
I chose JDO because the GAE persistence layer is built using datanucleus which uses JDO as its core api. JPA is implemented by datanucleus as an extension but my initial tests gave me a lot of problems that were hard to resolve due to the uninformative error messages.
A note, I found datanucleus error messages about mistakes in defining relationships etc. were extremely poor compared to hibernate and I hope this is an area they improve in.

Limitations



  • GAE doesn't support JNDI and spring's component scan autowiring tries to use it

  • Limitations on JDO implementation including no inheritence, bad support for un-owned relationships and only objects in the same group can be saved in one transactions



Step 1. Add necessary dependencies to maven project.


For a maven project this is simply a matter of declaring the dependencies in your pom.


<!--app engine related dependencies-->
<dependency>
<groupid>com.google.appengine</groupid>
<artifactid>appengine-api-1.0-sdk</artifactid>
<version>${appengine.version}</version>
</dependency>
<dependency>
<groupid>com.google.appengine.orm</groupid>
<artifactid>datanucleus-appengine</artifactid>
<version>${datanucleus-appengine.version}</version>
</dependency>
<dependency>
<groupid>javax.jdo</groupid>
<artifactid>jdo2-api</artifactid>
<version>${jdo2-api.version}</version>
</dependency>
<dependency>
<groupid>org.datanucleus</groupid>
<artifactid>datanucleus-core</artifactid>
<version>${datanucleus.version}</version>
</dependency>
<dependency>
<groupid>org.datanucleus</groupid>
<artifactid>datanucleus-jpa</artifactid>
<version>1.1.5</version>
</dependency>
<!--jstl -->
<dependency>
<groupid>javax.servlet</groupid>
<artifactid>jstl</artifactid>
<version>1.1.2</version>
</dependency>
<dependency>
<groupid>taglibs</groupid>
<artifactid>standard</artifactid>
<version>1.1.2</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-core</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-web</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-beans</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-orm</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-aop</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>org.springframework</groupid>
<artifactid>spring-jdbc</artifactid>
<version>2.5.6</version>
</dependency>
<dependency>
<groupid>cglib</groupid>
<artifactid>cglib</artifactid>
<version>2.2</version>
</dependency>

The data nucleus dependencies aren't in the central repo so you should add the following repo to you're pom

<pluginRepository>
<id>datanucleus</id>
<url>http://www.datanucleus.org/downloads/maven2</url>
</pluginRepository>


Step 2. Configure Spring Application Context


Setting up spring is quite simple, the only issue that needs a work around is that the annotation config used by spring to automagically inject depends on JNDI even if you're not using it. This causes as ClassNotFoundException to be thrown when deploying to the live GAE.
The simple work around for this is to declare internalPersistenceAnnotationProcessor bean in your config which tricks spring into thinking this class is already loaded and doesn't attempt to load it. I'm not totally happy with this work around but it works for the moment.

Create an applicationContext.xml in the class path as below.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
xmlns:tx="http://www.springframework.org/schema/tx" xmlns:context="http://www.springframework.org/schema/context"
xmlns:util="http://www.springframework.org/schema/util" xmlns:p="http://www.springframework.org/schema/p"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.5.xsd">

<!-- workaround appengine lack of JNDI support -->
<bean id="org.springframework.context.annotation.internalPersistenceAnnotationProcessor"
class="java.lang.String" />
<!-- End workaround -->

<context:component-scan base-package="com.agilewombat"/>
<context:annotation-config />

<tx:annotation-driven />

<!-- JDO Specific -->
<bean id="persistenceManagerFactory"
class="org.springframework.orm.jdo.LocalPersistenceManagerFactoryBean">
<property name="persistenceManagerFactoryName" value="transactions-optional" />
</bean>

<bean id="transactionManager" class="org.springframework.orm.jdo.JdoTransactionManager">
<property name="persistenceManagerFactory" ref="persistenceManagerFactory" />
</bean>
<!-- End JDO Specific -->
</beans>


Add spring to your web.xml


<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>classpath:applicationContext*.xml</param-value>
</context-param>

<!-- spring context loader lister -->
<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>



Step 3. Configure JDO



This is a standard JDO config. Create a jdoconfig.xml file in the META-INF directory of your resources.


<?xml version="1.0" encoding="utf-8"?>
<jdoconfig xmlns="http://java.sun.com/xml/ns/jdo/jdoconfig"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://java.sun.com/xml/ns/jdo/jdoconfig">

<persistence-manager-factory name="transactions-optional">
<property name="javax.jdo.PersistenceManagerFactoryClass"
value="org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory"/>
<property name="javax.jdo.option.ConnectionURL" value="appengine"/>
<property name="javax.jdo.option.NontransactionalRead" value="true"/>
<property name="javax.jdo.option.NontransactionalWrite" value="true"/>
<property name="javax.jdo.option.RetainValues" value="true"/>
<property name="datanucleus.appengine.autoCreateDatastoreTxns" value="true"/>
</persistence-manager-factory>
</jdoconfig>


Step 4. Generic Dao


You can write a dao that can be use for any of your domain objects by wrapping the spring JDO Template. This will also help you if you decide to switch to JPA. One major limitation of GAE is that transactions can only work on objects in the same group. This is typically will mean one object and its children. As such its not very usefuly to have transactional boundaries on the service layer as typical because you can't do much more than dao operations in a transaction. Instead transactions can be placed on the dao.


@Repository
@Transactional(readOnly = false)
public class GenericDao {

@Autowired
private PersistenceManagerFactory pmf;

private JdoTemplate jdoTemplate;

@PostConstruct
void createJdoTemplate() {
jdoTemplate = new JdoTemplate(pmf);
}

@Transactional(readOnly = true)
public <T extends DomainObject> T findByKey(Class<T> clazz, Key id) {
@SuppressWarnings("unchecked")
T entity = (T) jdoTemplate.getObjectById(clazz, id);

if (entity == null) {
throw new ObjectRetrievalFailureException(clazz, id);
}
return entity;
}

@Transactional(readOnly = true)
@SuppressWarnings("unchecked")
public <T extends DomainObject> Collection<T> findAll(Class<T> clazz) {
return jdoTemplate.detachCopyAll(jdoTemplate.find(clazz));
}

@Transactional
@SuppressWarnings("unchecked")
public <T extends DomainObject> void remove(T domainObj) {
T domainObject = (T) jdoTemplate.getObjectById(domainObj.getClass(),
domainObj.getKey());
jdoTemplate.deletePersistent(domainObject);
}

@Transactional(readOnly = true)
@SuppressWarnings("unchecked")
public <T extends DomainObject> Collection<T> findByNamedQuery(Class<T> clazz, String namedQuery, Map<String, Object> values) {
return jdoTemplate.findByNamedQuery(clazz, namedQuery, values);
}


@Transactional
@SuppressWarnings("unchecked")
public <T extends Object> T save(T domainObj) {
return (T) jdoTemplate.makePersistent(domainObj);
}

@Transactional
public void flush() {
jdoTemplate.flush();
}

}

Step 5. Ready To Go - Write Some Domain Objects


Now this is where the fun starts. The following annoying you should be aware of for those coming from hibernate.

  • Inheritence plain doesn't work so avoid it (This is a known bug in the gae layer that should be fixed in next release)

  • Un-owned relationships are not supported. This means to hold a reference to an object that is not a child you can only keep the primary key of the object and must manually dereference it in code.



An example User Class


@PersistenceCapable(identityType = IdentityType.APPLICATION, detachable = "true")
@Queries({
@Query(name = "findByUsername", value = "SELECT FROM User u where u.username == :username")
})
public class User {
private static final long serialVersionUID = 1L;

@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;

@Persistent
private String username;

@Persistent
private Date createdDate;

@Persistent
private Date modifiedDate;

@Persistent(defaultFetchGroup = "true")
private Key createdUser;

@Persistent(defaultFetchGroup = "true")
private Key modifiedUser;

@Persistent
private String password;

@Persistent
private Role role;

...
}


Step 6. Testing



Spring tests can be used as normal, the only thing extra that is required is to setup a base GAE environment. You can extend the following BaseTestFixture in your tests.


public class BaseTestFixture {

private boolean cleanEnvironment = true;

@Before
public void setUp() {
if (cleanEnvironment) {
ApiProxyLocalImpl proxy = new ApiProxyLocalImpl(new File(".")) {};
proxy.setProperty(LocalDatastoreService.NO_STORAGE_PROPERTY, Boolean.TRUE.toString());
ApiProxy.setDelegate(proxy);
ApiProxy.setEnvironmentForCurrentThread(new GoogleAppEngineTestEnvironment());
}
}

@After
public void tearDown() {
if (cleanEnvironment) {
ApiProxy.clearEnvironmentForCurrentThread();
}
}
}

Sunday, December 6, 2009

Code Syntax Highlighting

Any good code blog needs syntax highlighting. Unfortunately, blogger doesn't seem to offer this out of the box. Fortunately, there is a great little javascript package called SyntaxHighlighter written by Alex Gorbatchev here. Hosting for the syntax highlighter javascript is provided on his site so installing it for use in blogger is simply done by editing your blog template's html and inserting the javascript declarations below within the head element.


<link href='http://alexgorbatchev.com/pub/sh/2.1.364/styles/shCore.css' rel='stylesheet' type='text/css'/>
<link href='http://alexgorbatchev.com/pub/sh/2.1.364/styles/shThemeDefault.css' rel='stylesheet' type='text/css'/>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shCore.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushCpp.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushCSharp.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushCss.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushJava.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushJScript.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushPhp.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushPython.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushRuby.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushSql.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushVb.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushXml.js' type='text/javascript'></script>
<script src='http://alexgorbatchev.com/pub/sh/2.1.364/scripts/shBrushPerl.js' type='text/javascript'></script>
<script language='javascript'>
SyntaxHighlighter.config.bloggerMode = true;
SyntaxHighlighter.config.clipboardSwf = 'http://alexgorbatchev.com/pub/sh/2.1.364/scripts/clipboard.swf';
SyntaxHighlighter.all();
</script>



You can then wrap your code snippet in a <pre> tag in your code but this isn't perfect because html characters need to be escaped.


<pre class="brush: html">

</pre>



As an alternative you are meant to be able to can use the CDATA method below where html characters don't need to be escaped but it doesn't work for me as blogger doesn't let me post it.


<script class="brush: html" type="syntaxhighlighter"><![CDATA[...]]></script>

Saturday, December 5, 2009

First Post!

Lately I've been working on a couple of projects to experiment with different technologies I don't normally have much use for in my day job (e.g. GAE). As with anything new I've hit a few problems, some just noob mistakes, some limitations and some straight out bugs. In a few instances I've been really frustrated by the lack of information I could find googling and I've ended up with a lot of little bits of information rattling around in my head. So I decided to start this blog to help collect my thoughts on what worked/didn't work and hopefully help others who hit similar problems.