Spring Polyglot Persistent Applications Part 1 - Archived

From today I am going to kick off a blog series on how you can create polyglot persistence applications using Spring framework and deploy them on OpenShift platform as a service. The goal of this series is to build an application which uses both MySQL and MongoDB for storing data. The application will be built using current and latest release of Spring framework and will use other projects under Spring portfolio like Spring Security, Spring MongoDB, Spring Social, etc. But before we start developing application we need to understand the meaning of polyglot persistence and its need.

Setting up Expectation for Part 1

By the end of this blog-cum-tutorial you will have a Spring MongoDB application running on OpenShift. You will also know about terms like Polyglot Persistence, NoSQL,etc. and why time has come to think beyond RDBMS for data storage solutions. You will get your hands dirty with OpenShift and can deploy your own applications to OpenShift.

Framework Version

This blog will use the latest versions of frameworks.

  1. Spring Framework = 3.1.2 RELEASE
  2. Spring MongoDB = 1.1.0.M1

What is Polyglot Persistence?

The best definition that I have found for Polyglot Persistence on the web is by Martin Fowler in his article on Polyglot Persistence. I highly recommend reading Martin blog.

Polyglot Persistence is using multiple data storage technologies, chosen based upon the way data is being used by individual applications or components of single application.

Why can’t we only use RDBMS?

So far Relational databases have served us very well and they are tested in real environments and under real work load. RDBMS have very good tooling and framework support and ACID semantics are well understood by their huge developer community. But things have started to change, and we are struck by some of RDBMS limitations. Some of the limitations of RDBMS are :

  1. Complex Object Graphs lead to Joins which makes them difficult to scale horizontally.
  2. It is very difficult to evolve schema with time
  3. Inability to handle Semi-structured data
  4. Difficult to scale horizontally
  5. They are treated as one solution for every problem.

Do we have choices?

Gone are the days when we were struck with only RDBMS as data storage choice. Now there exists lot of non-relational databases labelled under NoSQL datastores. They are broadly classified into four categories.

  1. Document Data stores : These include MongoDB, CouchDB etc.
  2. Graph Data stores : These include Neo4j, Pregel, FlockDB etc.
  3. Key Value Data stores : These include Redis, Riak, LevelDB etc.
  4. Columnar Data stores : These include HBase, Cassandra etc.

Some of the NoSQL data stores are shown below.

The best thing about NoSQL data stores is that they are good for specific use case and solve a particular problem. So, you choose them depending on your use case. You do not have to commit to using a single data store for your entire application. Now that I have briefly introduced you to Polyglot Persistence lets talk about the application we will be developing in this blog series.

Application Use Case — Evernote clone

In this blog series we will be developing clone of Evernote application. For those of you who have not used Evernote, it is an application which stores your notes online. Evernote tech blog mentions that they use MySQL to store their data. The three main entities in their schema are Notebooks, Notes, and Users. The notes table has foreign key constraint on notebook table. This is shown below in an image.

In our application we will be storing Notebooks and Notes in MongoDB and Users data in MySQL. MongoDB does not support joins so we will be storing notes as an embedded array inside a Notebook. The users will still be stored in MySQL database. This is shown below.

The json document representing a notebook with embedded notes is shown below.

{
    "name" : "MongoDB Notebook",
    "description" : "My MongoDB Notes",
    "created" : ISODate("2012-09-25T12:22:23.590Z"),
    "author" : "test_user",
    "tags" : [
        "mongodb",
        "nosql"
    ],
    "notes" : [
        {
            "title" : "Say hello to MongoDB",
            "text" : "Say hello to MongoDB",
            "created" : ISODate("2012-09-25T12:22:23.593Z"),
            "tags" : [
                "mongodb",
                "getting-started"
            ]
        },
        {
            "title" : "How to setup ReplicaSet in MongoDB",
            "text" : "How to setup ReplicaSet in MongoDB",
            "created" : ISODate("2012-09-25T12:22:23.593Z"),
            "tags" : [
                "mongodb",
                "replication"
            ]
        }
    ]
}

Now that we know that we are developing a Notebook application for storing our notes. Lets start the development.

Prerequisite

  1. Install the OpenShift rhc client tools for your operating system by referring to this post https://www.openshift.com/developers/install-the-client-tools
  2. Download Eclipse from http://www.eclipse.org/downloads/
  3. Install the m2e (Maven integration for Eclipse) plugin from eclipse marketplace.
  4. Download and install git from http://git-scm.com/book/en/Getting-Started-Installing-Git
  5. It is optional to download MongoDB as we will be deploying application to OpenShift but in case you want to test your application locally you can download it from http://www.mongodb.org/downloads

Sign up for OpenShift

In case you don’t have OpenShift account you can create an OpenShift account at https://www.openshift.com/app/account/new. It will not more that 60 seconds to create an account. Its so fast.

Setting up OpenShift

The rhc client tool makes it very easy to setup your openshift instance with ssh keys, git and your applications namespace. The namespace is a unique name per user which becomes part of your application url. For example, if your namespace is cix and application name is notebook then url of the application will be https://notebook-cix.rhcloud.com/. The command is shown below.

rhc setup -l <openshift_login_email>

Source code of Notebook Application

Although I will be covering a step by step guide on how to create the application. In case you just want to take a look at code. You can get the code from github https://github.com/shekhargulati/notebook-part1

Creating Notebook Application

After setting up the domain name the next step is to create the application. To create an application type the following command.

rhc app create notebook jbossas-7 -l <openshift_login_email> -d

The -d option tells the rhc client to print the debug information on the console. This is very helpful in case something goes wrong while creating application instance. You can specify -d option command with all the commands. You can view the application online at https://notebook-cix.rhcloud.com/. Almost all the rhc commands will require you to provide valid email and password. You can avoid entering login email by specifying a default login in express.conf file which exists under user.home/.openshift folder as shown below.

default_rhlogin='sgulati@redhat.com'

To avoid entering password with each command you can follow this blog by Sumana. But it will only work for linux users. The most important artifact generated by rhc create command is pom.xml. The pom.xml contains a maven profile called openshift which is invoked by default when we push the code to OpenShift. Please make sure that your maven project always have openshift profile otherwise code will not get deployed.

Adding MongoDB and RockMongo Client Cartridge

This application will use MongoDB for persistence so we should add MongoDB cartridge support to it. Optionally you can also add support for RockMongo which is a client application like phpMyAdmin but for MongoDB. You can add cartridges using following commands.

rhc cartridge add mongodb-2.2 -a notebook
rhc cartridge add rockmongo-1.1 -a notebook

Please note down the information returned by server for the cartridges.

Creating multi module maven project

After we have created the application using rhc app createcommand we will create two maven modules using m2e plugin. To do this import the notebook project as maven project in eclipse and change the packaging to pom in pom.xml. The snippet is shown below.

<modelVersion>4.0.0</modelVersion>
<groupId>com.openshift.notebook</groupId>
<artifactId>notebook</artifactId>
<packaging>pom</packaging>
<version>1.0</version>
<name>notebook</name>

Right click on the notebook project and create two maven modules — notebook-core and notebook-web. The project structure will look like as shown below.
The notebook-core module will contain classes related to domain model, repositories, and service and notebook-web will contain Spring MVC controller classes and other UI related stuff.

To verify that everything works fine lets push the code to git. To do that type in following git commands.

git add .
git commit -a -m "maven multi module project structure created"
git push

You should be able to see the default application running at https://notebook-cix.rhcloud.com/.

Creating Document Model

Now that we have our maven module structure ready lets create the document model. For this application we will have two domain objects Notebook and Note. Note will not be a separate collection but will exist as a embedded document inside Notebook. In this application we are using Spring MongoDB for working with MongoDB. The Notebook and Note classes are shown below. For brevity I have removed set and get methods.

@Document(collection = "notebooks")
public class Notebook {
 
    @Id
    private String id;
 
    private String name;
 
    private String description;
 
    @DateTimeFormat(style = "M-")
    private Date created = new Date();
 
    private String author;
 
    private String[] tags;
 
    private Note[] notes;

The Note class is shown below.

public class Note {
 
    private String id;
 
    private String title;
 
    private String text;
 
    @DateTimeFormat(style = "M-")
    private Date created = new Date();
 
    private String[] tags;

Important things in the code snippet above are :

  1. @Document annotation identifies a domain object to be persisted in MongoDB. The notebooks specify the name of collection which will be created in MongoDB.
  2. @Id annotation marks this field as Id field which will be auto generated by MongoDB.
  3. The Notebook document showcase the richness of MongoDB documents. It shows that you can store fields as arrays, Object arrays i.e. notes , String, number, date etc.

Persisting into MongoDB — Concept of Repositories

Spring MongoDB has concept of repositories where in you implement interface at Spring will generate a proxy class with the implementation. This makes it very easy to write repository classes and removes lots of boiler plate code. You can read more about Spring MongoDB in its documentation. Lets create our NotebookRepository as shown below.

import java.util.List;
 
import org.springframework.data.repository.PagingAndSortingRepository;
import org.springframework.stereotype.Repository;
 
import com.openshift.notebook.core.domain.Notebook;
 
@Repository
public interface NotebookRepository extends
        PagingAndSortingRepository<Notebook, String> {
 
    List<Notebook> findAll();
 
}

The important things in the code snippet shown above are :

  1. NotebookRepository interface extends PagingAndSortingRepository interface which defines CRUD methods and finder methods supporting pagination and sorting. So, the proxy generated by Spring will have all those methods.
  2. @Repository annotation is a specialization of @Component annotation which indicates that class is a repository or DAO class. A class annotated with @Repository is eligible for Spring DataAccessException translation when used in conjunction with a PersistenceExceptionTranslationPostProcessor.

Configuring Repositories

Spring will not generate proxies for repositories by default. It is your resposibility to specify that in a Spring context file as shown below.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:mongo="http://www.springframework.org/schema/data/mongo"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd        http://www.springframework.org/schema/data/mongo        http://www.springframework.org/schema/data/mongo/spring-mongo-1.0.xsd        http://www.springframework.org/schema/beans        http://www.springframework.org/schema/beans/spring-beans-3.1.xsd">
 
    <mongo:repositories base-package="com.openshift.notebook.core.repository" />
 
    <context:annotation-config />
 
</beans>

The mongo:repositories tag tell Spring to generate proxies for classes in some package.

Configuring MongoDBFactory and MongoTemplate

Now that we have done Spring configuration for repositories the next step is to define MongoDBFactory which is factory for creating DB instances. We can configure the MongoDBFactory in XML using mongo namespace but I will do that using Spring 3 Bean annotation support. Create a class MongoDBConfig as shown below.

@Configuration
@ImportResource("classpath:applicationContext-mongo.xml")
public class MongoDbConfig {
 
    @Bean
    public MongoTemplate mongoTemplate() throws Exception {
        MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory());
        return mongoTemplate;
    }
 
    @Bean
    public MongoDbFactory mongoDbFactory() throws Exception {
        Mongo mongo = new Mongo("localhost", 27017);
        String databaseName = "notebook";
        MongoDbFactory mongoDbFactory = new SimpleMongoDbFactory(mongo,
                databaseName);
        return mongoDbFactory;
    }
 
}

But the code listing shown above has limitation that when we deploy code to OpenShift it will throw an exception because MongoDB configuration will be different on MongoDB running in cloud. To overcome this limitation we can make use of Spring 3.1 feature called Profile which allow us to have different bean defintion under different environment. For example, for development we would want to work against our local MongoDB instance but on OpenShift we would like to use MongoDB running on OpenShift. The solution is to two different Configuration classes one which will create MongoDBFactory for local development and other for OpenShift. The two classes are shown below. Please note that they both implement same interface and we inject that interface in MongoDBConfig class.

Development MongoDBFactory

@Configuration
@Profile("dev")
public class DevMongoDBFactoryConfig implements MongoDbFactoryConfig {
 
    /* (non-Javadoc)
     * @see com.openshift.notebook.core.config.MongoDbConfig#mongoDbFactory()
     */
    @Override
    @Bean
    public MongoDbFactory mongoDbFactory() throws Exception {
        Mongo mongo = new Mongo("localhost", 27017);
        String databaseName = "notebook";
        MongoDbFactory mongoDbFactory = new SimpleMongoDbFactory(mongo,
                databaseName);
        return mongoDbFactory;
    }
 
}

OpenShift MongoDBFactory

@Configuration
@Profile("openshift")
public class OpenShiftMongoDBFactoryConfig implements MongoDbFactoryConfig {
 
    @Override
    public MongoDbFactory mongoDbFactory() throws Exception {
        String openshiftMongoDbHost = System.getenv("OPENSHIFT_MONGODB_DB_HOST");
        int openshiftMongoDbPort = Integer.parseInt(System.getenv("OPENSHIFT_MONGODB_DB_PORT"));
        String username = System.getenv("OPENSHIFT_MONGODB_DB_USERNAME");
        String password = System.getenv("OPENSHIFT_MONGODB_DB_PASSWORD");
        Mongo mongo = new Mongo(openshiftMongoDbHost, openshiftMongoDbPort);
        UserCredentials userCredentials = new UserCredentials(username,password);
        String databaseName = System.getenv("OPENSHIFT_APP_NAME");
        MongoDbFactory mongoDbFactory = new SimpleMongoDbFactory(mongo, databaseName, userCredentials);
        return mongoDbFactory;
    }
 
 
}

Correspondingly MongoDBConfig class will also change as shown below.

@Configuration
@ImportResource("classpath:applicationContext-mongo.xml")
public class MongoDbConfig {
 
    @Inject
    private MongoDbFactoryConfig mongoDbFactoryConfig;
 
    @Bean
    public MongoTemplate mongoTemplate() throws Exception {
        MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactoryConfig.mongoDbFactory());
        return mongoTemplate;
    }
 
}

Depending on the active profile either DevMongoDBFactoryConfig or OpenShiftMongoDBFactoryConfig will be instantiated and bean will be injected in MongoDBConfig.

Lets test the existing code using a JUnit test case. The code snippet is shown below which activates dev profile so the application uses MongoDB running on local system. To run this test you need to spring-test maven dependency.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = { DevMongoDBFactoryConfig.class,MongoDbConfig.class })
@ActiveProfiles("dev")
public class NotebookRepositoryTest {
 
    @Inject
    NotebookRepository notebookRepository;
 
    @Inject
    MongoTemplate mongoTemplate;
 
    @Before
    public void setup() {
        mongoTemplate.dropCollection(Notebook.class);
    }
 
    @Test
    public void shouldSaveNotebookAndReturnCountAsOne() {
        notebookRepository.save(createNewNotebook());
        assertEquals(1, notebookRepository.count());
    }
}

The last thing remaining in the core module is service class which just acts as a delegate to repository class. You can checkout the class in github.

Creating Rest Controller

The last step for this blog is to publish a REST Json web service for performing CRUD operations on Notebook. To do that, we will create a Spring MVC controller with CRUD operations as shown below. The controller exposes CREATE, GET,POST, DELETE operations.

@Controller
@RequestMapping("/notebooks")
public class NotebookController {
 
    @Inject
    NotebookService notebookService;
 
    @RequestMapping(method = RequestMethod.POST, headers = "Accept=application/json")
    public ResponseEntity<String> createFromJson(@RequestBody String json) {
        Notebook notebook = Notebook.fromJsonToNotebook(json);
        notebookService.createNewNotebook(notebook);
        HttpHeaders headers = new HttpHeaders();
        headers.add("Content-Type", "application/json");
        return new ResponseEntity<String>(notebook.getId(),headers, HttpStatus.CREATED);
    }
 
    @RequestMapping(value = "/{id}", headers = "Accept=application/json")
    @ResponseBody
    public ResponseEntity<String> showJson(@PathVariable("id") String id) {
 
        Notebook notebook = notebookService.findNotebook(id);
        HttpHeaders headers = new HttpHeaders();
        headers.add("Content-Type", "application/json; charset=utf-8");
        if (notebook == null) {
            return new ResponseEntity<String>(headers, HttpStatus.NOT_FOUND);
        }
        return new ResponseEntity<String>(notebook.toJson(), headers,
                HttpStatus.OK);
    }
 
    @RequestMapping(headers = "Accept=application/json")
    @ResponseBody
    public ResponseEntity<String> listJson() {
 
        HttpHeaders headers = new HttpHeaders();
        headers.add("Content-Type", "application/json; charset=utf-8");
        List<Notebook> result = notebookService.findAllNotebooks();
        return new ResponseEntity<String>(Notebook.toJsonArray(result),
                headers, HttpStatus.OK);
    }
 
    @RequestMapping(value = "/{id}",method = RequestMethod.PUT, headers = "Accept=application/json")
    public ResponseEntity<String> updateFromJson(@PathVariable("id") String id,@RequestBody String json) {
        HttpHeaders headers = new HttpHeaders();
        headers.add("Content-Type", "application/json");
        Notebook notebook = Notebook.fromJsonToNotebook(json);
        notebook.setId(id);
        if (notebookService.updateNotebook(notebook) == null) {
            return new ResponseEntity<String>(headers, HttpStatus.NOT_FOUND);
        }
        return new ResponseEntity<String>(headers, HttpStatus.OK);
    }
 
    @RequestMapping(value = "/{id}", method = RequestMethod.DELETE, headers = "Accept=application/json")
    public ResponseEntity<String> deleteFromJson(@PathVariable("id") String id) {
        Notebook notebook = notebookService.findNotebook(id);
        HttpHeaders headers = new HttpHeaders();
        headers.add("Content-Type", "application/json");
        if (notebook == null) {
            return new ResponseEntity<String>(headers, HttpStatus.NOT_FOUND);
        }
        notebookService.deleteNotebook(notebook);
        return new ResponseEntity<String>(headers, HttpStatus.OK);
    }
}

Gluing it all together and deploying to OpenShift
After you have created the application using rhc create app command and added MongoDB and RockMongoDB client cartridge using rhc app cartridge add command you have to checkout the code from my github. To do that follow the steps mentioned below.

git rm -rf src/ pom.xml
 
git commit -am "removed default files"
 
git remote add notebook -m master git://github.com/shekhargulati/notebook-part1.git
 
git pull -s recursive -X theirs notebook master
 
git push

Testing the REST web services

To test the REST web service I will use curl to create, read, update and delete notebook.

To create the Notebook with notes the curl command is shown below.

curl -i -X POST -H "Content-Type: application/json" -H "Accept: application/json" -d  '{"author":"test_user","description":"this is a test notebook","name":"test notebook","notes":[{"tags":["test","xyz"],"text":"test note 1","title":"test note title"},{"tags":["test","xyz"],"text":"test note 2","title":"test note title"}],"tags":["test","xyz"]}' https://notebook-cix.rhcloud.com/notebooks

To read the Notebook by id curl command is shown below. To find the id of the persisted Notebook login to RockMongo client and go to the notebooks collection as shown below.

curl -i -H "Accept: application/json" https://notebook-cix.rhcloud.com/notebooks/50618372e4b02daab4d93960

To update the Notebook for id curl command is shown below.

curl -i -X PUT -H "Content-Type: application/json" -H "Accept: application/json" -d '{id="50618372e4b02daab4d93960","author":"test_user","created":1348554253996,"description":"this is a test notebook123445","name":"test notebook12345","tags":["test","xyz"]}' https://notebook-cix.rhcloud.com/notebooks

To delete the Notebook with id curl command is shown below.

curl -i -X DELETE -H "Accept: application/json" https://notebook-cix.rhcloud.com/notebooks/50618372e4b02daab4d93960

Conclusion

In this blog I talked about Polyglot persistence and how you can create a Spring MongoDB application and deploy it on OpenShift. This blog showcase that all the new features and releases of Spring frameworks works like charm on OpenShift.

In the next part of this series we will setup Jenkins and incorporate User in the application.

What’s Next?

Categories
MongoDB, MySQL, OpenShift Online
Tags
, ,
Comments are closed.