9781449323936
chapter_persistence.html

Chapter 3. Persistence

The persistence strategy in Grails is called GORM, Grails Object Relational Mapping. In earlier versions of Grails this was a wrapper around Hibernate’s ORM implementation, but has been abstracted to allow access to other data sources, including NoSQL datastores like MongoDB, Redis, and Neo4j. GORM for Hibernate has been retrofitted to follow the new GORM API, which provides a consistent developer experience regardless of the underlying storage implementation.

But it’s not merely a least-common-denominator approach where only the features that are common to all datastores are supported. Instead there is a core set of functionality that all implementations must provide (mapping from domain classes to datastore storage, supporting dynamic finders and Criteria queries, etc.) but each provider also supports access to the underlying implementation to access non-core features directly.

This means that for data that fits the relational model well, you can use Hibernate to manage persistence in a relational database, but for less structured data or to support more dynamic schemas, you can use a NoSQL datastore. The Grails NoSQL plugins coexist well with Hibernate, so you’re free to store all of your data in a relational database, all in one or more NoSQL datastores, or use a mix.

Data Mapping

The convention for mapping a Grails class to a datastore representation is to create a class under the grails-app/domain directory. These can be as simple as a class with one or more public fields, for example this Person class

package com.acme

class Person {
   String firstName
   String lastName
}

You can create domain classes by hand with your IDE or text editor since they’re just Groovy classes in the grails-app/domain directory (in the subdirectory corresponding to their package) but the preferred way to create them is with the create-domain-class script since it uses a standard template and automatically creates a corresponding test class:

$ grails create-domain-class person

You can see from the output that it creates a test class as well as the domain class (take this as a strong hint to remember to write tests!)

| Created file grails-app/domain/appname/Person.groovy
| Created file test/unit/appname/PersonTests.groovy

This example will create the class in the default artifact package since one isn’t specified (this is determined by the grails.project.groupId property in Config.groovy). You can specify the package to override this behavior:

$ grails create-domain-class com.mycompany.Book

The domain class is created from a template which you can customize. If you find yourself copying and pasting previous domain classes to save yourself the typing for commonly-added features, or if you manually make these changes repeatedly, you can install the templates into your project and edit them:

$ grails install-templates

The template for domain classes is in src/templates/artifacts/DomainClass.groovy:

@artifact.package@class @artifact.name@ {

    static constraints = {
    }
}

If you have common code that you repeatedly add to new domain classes you should update the template instead and save yourself the time. For example you could make all new domain classes implement Serializable and add an empty mapping block:

@artifact.package@class @artifact.name@ implements Serializable {

   static constraints = {
   }

   static mapping = {
   }
}

Non-persistent domain classes

You may ask how to have non-persistent domain classes in your application, and the short answer is that you can’t. Of course everything is possible, but Grails assumes that if you are following the convention of putting a Groovy class in the grails-app/domain folder, you want the class to be persistent; you don’t have to configure this, so there’s no direct way to make it non-persistent. But it is possible to have classes that model domain objects that don’t map to a database table.

One way is to simply not invoke any GORM methods on domain classes that don’t use the database. In "create-drop" mode you will end up with database tables for these, but you’ll be switching to using database migrations soon enough and at that point you can just choose to not create the tables.

The other way is to put them in the src/groovy (or src/java). This has the benefit of being ignored by GORM, but it does partition your domain model into two different sections of your code.

The real issue here is a semantic one; the "domain" artifact type in Grails is for your persistent domain classes, and Grails doesn’t have a well-defined concept of a general domain model, leaving that up to you.

Data Validation

GORM supports data validation with the constraints DSL. This is declared as a static closure in your domain class containing the various rules that define what is and isn’t valid data for each domain class. There are several built-in constraints (blank, nullable, matches, creditCard, etc.) and for more complex validation checks there’s also a custom validation style. By default all persistent properties are considered to be not-null, so there is an implicit nullable: false constraint.

Table 3.1, “GORM Constraints” shows the built-in constraints.

Table 3.1. GORM Constraints

NameDescriptionI18N CodeNotes

blank

Whether an empty string is allowed

className.propertyName.blank

creditCard

Checks that a String is a valid credit card number

className.propertyName.creditCard.invalid

Uses the Apache Commons CreditCardValidator

email

Checks that a String is a valid email address

className.propertyName.email.invalid

Uses the Apache Commons EmailValidator

inList

Checks that the value is in the specified list

className.propertyName.not.inList

Make sure the type of the list values matches the property type; consider using an Enum if possible

matches

Checks that a String matches a regular expression

className.propertyName.matches.invalid

max

Checks that a Comparable property doesn’t exceed the specified value

className.propertyName.max.exceeded

maxSize

Checks the maximum size of a String, Collection, or array

className.propertyName.maxSize.exceeded

min

Checks that a Comparable property doesn’t exceed the specified value

className.propertyName.max.exceeded

minSize

Checks the minimum size of a String, Collection, or array

className.propertyName.maxSize.exceeded

notEqual

Checks that the value is not equal to the specified value

className.propertyName.notEqual

nullable

Checks that the value is not null

className.propertyName.nullable

Defaults to false

range

Checks that a Comparable property is within the specified Groovy range

className.propertyName.range.toosmall or className.propertyName.range.toobig

size

Restricts the size of a String, Collection, or array to the specified Groovy range

className.propertyName.size.toosmall or className.propertyName.size.toobig

unique

Checks that the value is unique

className.propertyName.unique

Can check one or multiple fields; executes a select query during validation

url

Checks that the value is a valid URL

className.propertyName.url.invalid

Uses the Apache Commons UrlValidator


In addition there are a two checks listed as constraints that are used to affect UI rendering but don’t register validation errors, attributes and widget.

The syntax for specifying constraints is fairly straightforward; add the static constraints block and inside it list the various data validation rules:

class Thing {
   String cardNumber
   String color
   String email

   static constraints = {
      cardNumber creditCard: true, blank: false
      color inList: ['Red', 'Green', 'Blue']
      email unique: true, size: 5..100
   }
}

Lets take a closer look at the DSL to see how this works. To a new user this syntax can seem a bit strange, but there’s not really that much magic here. Each line is simply a method call. But looking at the domain class, there’s obviously no cardNumber, color, or email methods, just properties. It’s more clear if we add in the optional parentheses:

static constraints = {
   cardNumber(creditCard: true, blank: false)
   color(inList: ['Red', 'Green', 'Blue'])
   email(unique: true, size: 5..100)
}

The first line is a call to a (nonexistent) cardNumber method, and the parameters are a single Map ([creditCard: true, blank: false]). This works because the Grails code that evaluates the constraints block registers an instance of org.codehaus.groovy.grails.validation.ConstrainedPropertyBuilder as the delegate of the closure, and then invokes the closure. Since it is valid Groovy code it runs, and missing methods and properties get dispatched to the builder which has the logic to convert the method calls and the Map values to constraint rules for the associated property (as long as there is a property corresponding to each method invocation). Each key/value pair in the paramter Map defines a constraint and its data. The constraints are implemented as classes implementing the Grails org.codehaus.groovy.grails.validation.Constraint interface, for example NullableConstraint, MaxConstraint, and InListConstraint.

Custom Validation

Grails comes with many of the standard checks that you will need but there will be cases where these checks aren’t sufficient. If you need to check a value in relation to another value or do a more complicated check, you can create a custom validator. These are defined as closures with the "validator" name:

class User {
   String username
   String password

   static constraints = {
      username blank: false, unique: true
      password blank: false, size: 8..100, validator: { pwd, user ->
         if (user.password == pwd) {
            return 'user.password.matchesUsername'
         }
      }
   }
}

A custom validator can have from zero to 3 parameters. If it declares one its value is the field value being validated (declaring none is the same, except that there is one parameter with default closure name it):

static constraints = {
   ...
   fieldName validator: { fieldValue ->
      // validate the field's value
   }
}

Note

In all of these examples, the parameter names are just examples and you can name them however you like; only the number of parameters is significant to GORM.

If a custom validator declares two parameters then the first is the value and the second is the domain class instance being validated. This variant is useful if you need to access other persistent values in the instance, and also to access other class data such as dependency-injected fields. For example, it’s a common practice to delegate business logic to services. By adding a dependency injection for the service we can access it from the current instance:

class User {

   def userService

   String username
   String password

   static constraints = {
      ...
      password blank: false, size: 8..100, validator: { pwd, user ->
         return user.userService.validatePassword(pwd, user)
      }
   }
}

If the validator has three parameters then the first is the value, the second is the instance, and the third is the Spring Errors instance. Use this approach when you need to directly call rejectValue or other Errors methods:

static constraints = {
   ...
   fieldName validator: { fieldValue, instance, errors ->
      ...
   }
}

Grails ignores any return value when using the three-parameter variant since it’s assumed that your validator works directly with the Errors instance.

Note

You can reuse custom validators by making them static properties of a utility class and referring to them from your domain classes:

class Validators {
   static passwordCheck = { pwd, user -> ... }
   static otherValidator = { value, obj, errors -> ... }
}
static constraints = {
   ...
   password blank: false, size: 8..100, validator: Validators.passwordCheck
}

If you need to accessing the database in a custom validator be sure to wrap the calls in a withNewSession block. If you don’t you run the risk (only in Hibernate currently) of the query triggering a flush and causing unexpected problems.

Extreme Custom Validation

To really take matters into your own hands you can completely bypass the standard validation approach and define your own validator class (or reuse an existing one, for example from a traditional Spring application). Each domain class artifact has a org.codehaus.groovy.grails.validation.GrailsDomainClassValidator. This is registered as a Spring bean with a bean name that’s the full class name with package and the suffix "Validator", so the com.myapp.User domain class would have a corresponding com.myapp.UserValidator validator bean. So to define your own, register it in grails-app/conf/spring/resources.groovy with the correct name and it will replace the one that Grails registers by default:

import com.myapp.MyUserValidator

beans = {
   'com.myapp.UserValidator'(MyUserValidator)
}

Put this class in src/groovy or src/java and implement the Spring org.springframework.validation.Validator interface. Grails validators support cascading to validate associated many-to-one fields; to participate in this implement org.codehaus.groovy.grails.validation.CascadingValidator instead. You’ll probably find that it’s most convenient to subclass org.codehaus.groovy.grails.validation.GrailsDomainClassValidator and override one or more of validate(Object obj, Errors errors), validate(Object obj, Errors errors, boolean cascade), and postValidate(Object obj, Errors errors).

If you just implement one of the validator interfaces and don’t subclass GrailsDomainClassValidator you’ll also need to wire up the validator to the domain class in BootStrap.groovy since this won’t be done automatically:

class BootStrap {

   def grailsApplication

   def init = { servletContext ->
      grailsApplication.getDomainClass('com.myapp.User').validator =
         grailsApplication.mainContext.getBean('com.myapp.UserValidator')
   }
}

Validation Plugins

There are also plugins available that offer custom validation. The extra-validators plugin has a validator for password confirmation, and one that checks post codes for the UK, the US, and Canada.

The constraints plugin does a great job of formalizing the process for creating custom constraints. Each validator is defined as a closure in its own class, and by convention the name of the class is used as the constraint type in the constraints block. For example if you have custom logic to validate a zip code, you can create a ZipCodeConstraint class in grails-app/utils and the plugin will make it available as zipCode:

class Address {
   String line1
   String line2
   String state
   String zip

   static constraints = {
      ...
      zip zipCode: true
   }
}

See the plugin documentation for more examples of how to use the plugin.

Friendly Error Messages

When debugging it’s common to render the validation errors, but you’ve probably found that the default toString representation of the Errors object isn’t very useful. Here’s a convenient way to add a getErrorStrings method to the MetaClass of all domain classes in BootStrap.groovy:

class BootStrap {

   def grailsApplication
   def messageSource

   def init = { servletContext ->

      for (dc in grailsApplication.domainClasses) {
         dc.metaClass.getErrorStrings = { Locale locale = Locale.getDefault() ->
            def stringsByField = [:]
            for (fieldErrors in delegate.errors) {
               for (error in fieldErrors.allErrors) {
                  def resolved = stringsByField[error.field]
                  if (!resolved) {
                     resolved = []
                     stringsByField[error.field] = resolved
                  }
                  resolved << messageSource.getMessage(error, locale)
               }
            }
            stringsByField
         }
      }
   }
}

It uses the messageSource bean to resolve the validation message from the appropriate messages.properties file for the specified Locale and returns a Map where the keys are field names and the values are a List of resolved validation messages. You can print all of the errors:

def person = new Person(...)
if (!person.save()) {
   log.debug "$person errors: $person.errorStrings"
}

or grab just the errors for a particular field:

def person = new Person(...)
if (!person.save()) {
   log.debug "$person username errors: $person.errorStrings.username"
}

Blanks vs Nulls

In many cases a blank string and null are equivalent; there’s no value set. But HTTP submissions from web browser POST requests send blank strings for inputs without a value. This won’t be the case with non-HTTP data such as from other external clients like web services or during testing, so converting blanks to nulls for the HTTP tier will help simplify validation. While we’re at it we can also trim extra whitespace from submitted values.

To do this, create a filter in whatever package makes sense for your application (or add this code to an existing filters class):

grails create-filters com.mycompany.myapplication.SiteFilters

The name of the filter isn’t important; it’s just there so each one is distinct. In this case we want to filter all requests but ignore GET requests since we’re only concerned with form submissions, so we check for GET or POST with the isPost() metamethod added to HttpServletRequest:

package com.mycompany.myapplication

class SiteFilters {

   def filters = {
      blankToNullAndTrim(controller: '*', action: '*') {
         before = {
            if (request.post) {
               convertBlanksToNullsAndTrim(params)
            }
            true
         }
      }
   }

   private static void convertBlanksToNullsAndTrim(Map map) {
      def keys = [] + map.keySet() // copy to avoid ConcurrentModificationException
      for (name in keys) {
         def value = map[name]
         if (value instanceof String) {
            value = value.trim()
            if (value.length() == 0) {
               map[name] = null // don't remove - explicity set to null
            }
            else {
               map[name] = value // update if trimmed
            }
         }
         else if (value instanceof Map) {
            // recurse with empty nested param, e.g. "location":["id":""]
            convertBlanksToNullsAndTrim value
         }
      }
   }
}

If you use this approach you can remove all of your blank: false constraints since you’ll never have a blank string, only null or a String with a non-zero length.

Transients

All typed public fields in a domain class are considered persistent. Any untyped fields (either typed as Object or declared with def) are ignored because GORM needs to know how to represent the values in the datastore. To indicate that a typed field shouldn’t be persisted, you can add it to the transients list:

class Video {
   String url
   Boolean viewed

   static transients = ['viewed']
}

Here url is persistent, but viewed isn’t - it might just be used to set a temporary value during the request.

Typed getter and setter methods can be considered to represent persistent properties. A getter without a corresponding setter or a setter without a corresponding getter are fine; these utility methods will be ignored by GORM and Hibernate. But if you create a matched pair of a getter and setter, they create a JavaBean property and are considered to be persistent. This is because public Groovy fields are converted by the Groovy compiler to a private field and a public getter/setter pair. GORM has no way of knowing that your getter/setter pair was created by you and not by the compiler. So if you need these, just add their corresponding property name to the transients list

class Person {
   String name
   private int shares

   int getShareCount() { shares }
   void setShareCount(int sc) { shares = sc }

   static transients = ['shareCount']
}

Mapping Collections

GORM uses mapped collections as its standard approach to mapping many-to-one and many-to-many relationships. For example a Purchase has many OrderItems, so this would be modeled as

class Purchase {
   Date purchaseDate
   static hasMany = [orderItems: OrderItem]
   // other properties
}

and

class OrderItem {
   String itemId
   Integer quantity
   static belongsTo = [purchase: Purchase]
   // other properties
}

and will generate database tables as described in the ER diagram in Figure 3.1, “Purchase and OrderItem ER Diagram”.

Figure 3.1. Purchase and OrderItem ER Diagram

Purchase and OrderItem ER Diagram

As modeled this is a bidirectional relationship, and the belongsTo property ensures that Purchase deletes cascade to the OrderItem instances. To associate an OrderItem with a Purchase, call the method addToOrderItems which is dynamically added to to the metaclass by GORM

def purchase = ...
purchase.addToOrderItems(new OrderItem(...))
purchase.save()

You could also use the simpler form

class OrderItem {
   String itemId
   Integer quantity
   static belongsTo = Purchase
   // other properties
}

which would use a join table instead of a foreign key from the order_item table to the purchase table.

Unfortunately while this is convenient, it isn’t the best approach for performance. This is because the hasMany declaration adds a Set property containing the OrderItem instances. A Set guarantees uniquness, and to enforce this Hibernate (or whichever datastore you’re using) has to load all of the other instances from the database to check whether the new instance is already there (or equivalent to one based on hashCode and equals checks). The collection defaults to being lazy-loaded, but accessing its contents triggers a full load from the database.

The problem is the same if you change the collection type to List

class Purchase {
   Date purchaseDate
   List orderItems
   // other properties
   static hasMany = [orderItems: OrderItem]
}

since the List order must be maintained, so again all of the existing instances will be loaded.

If you’re using Hibernate and enable SQL logging, either to stdout with the logSql property in DataSource.groovy

dataSource {
   dbCreate = ...
   url = ...
   ...
   logSql = true
}

or with the org.hibernate.SQL Log4j logger in Config.groovy

log4j = {
    error 'org.codehaus.groovy.grails',
          'org.springframework',
          'org.hibernate',
          'net.sf.ehcache.hibernate'

    debug 'org.hibernate.SQL'
    // trace 'org.hibernate.type'
}

you’ll see all the unexpected database activity.

The underlying implementation of the persistent collections are dirty-aware Hibernate classes (or analagous implementations for your NoSQL implementation). So when you add one new instance to a collection Hibernate is aware of that change, and when the class is saved the child object will be too. But by inverting the ownership (in traditional Hibernate applications collections are usually "inverse" collections that don’t drive persistence) we’ve added these extra collection maintentance costs.

In addition, even though we’re just attempting to store a new OrderItem instance, since the collection is a property of the Purchase class, the version of the Purchase will be incremented. This means that concurrent changes run a fairly high risk of an optimistic locking exception for the Purchase.

Mapping a many-to-many relationship is similar. A common example is the User and Role relationship when modeling security

class User {
   String username
   String password

   static hasMany = [roles: Role]
}
class Role {
   String name

   static hasMany = [users: User]
   static belongsTo = User
}

Changing the type to a Bag (which has no uniqueness or order guarantees) would seem to be solution, but it still has issues; see my "Hibernate Bags in Grails 2.0" blog post http://burtbeckwith.com/blog/?p=1029

class Purchase {
   Date purchaseDate
   Collection orderItems
   static hasMany = [orderItems: OrderItem]
   // other properties
}

similarly doesn’t help, at least not enough.

The Solution

I described this issue at SpringOne/2GX in 2010; you can watch the talk online at http://www.infoq.com/presentations/GORM-Performance and you can also see this discussed in my http://burtbeckwith.com/blog/?p=169 blog post which addresses these issues and has sample code and a PDF of an earlier presentation.

Once you have determined that the number of elements in a mapped collection will be a performance concern (there is no standard number for this - it depends on your use cases) you can address the issue by simply not using mapped collections. You will lose some convenience, but it is relatively easy regain a lot of it.

The solution for the OrderItem/Purchase many-to-one mapping is to remove the hasMany in Purchase, replace the belongsTo in OrderItem with a reference to the owning Purchase, for example Purchase purchase. This will result in the same database structure (unless you were using a join table which is rare in many-to-one) since there is still a foreign key from the order_item table to the purchase table. Saving an OrderItem instance changes, from adding it to the orderItems collection in Purchase to setting the purchase reference in the OrderItem and saving it directly. This will be more efficient and is arguably more intuitive.

One thing that is lost in this approach is a convenient way to get all of the OrderItem instances associated with a Purchase, but that’s easy to get back: add a method in the Purchase class that returns OrderItem.findAllByPurchase(this). You also lose cascaded deletes if you had specified the dependsOn property, but it’s simple enough to delete a Purchase in a transactional service method that deletes the associated OrderItem instances first.

The fix for many-to-many is more involved, but not too bad. Ordinarily Grails developers don’t think much about the join table that links the two "many" tables since it’s transparent in the code. But you can map a domain class to that table, and it just needs two properties, foreign keys for each of the other tables. If the primary key is defined as composite and comprised of these two foreign keys, the table structure will be the same as what Grails was using; this makes data migration a no-op. There is a bit more work involved since Hibernate requires that the join table domain class implement Serializable and have well-defined equals and hashCode methods. You can see an example of this in the above referenced presentation, and if you’re using the spring-security-core plugin you already have an example in your application since the UserRole class uses exactly this approach.

Site last updated on: December 11, 2012 at 11:47:18 AM PST