Spring Boot + Spring Batch + MySQL

Overview

In this article, we'll explore how to integrate Spring Boot with Spring Batch and MYSQL.

As we have seen Overview/Architecture of Spring Batch in previous article.

Now, let's create simple Spring Batch with Spring Boot and MySQL Application.

Project Structure

Create Spring Boot Maven project

Add the spring-boot-starter-batch and mysql-connector-java dependencies in pom.xml.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.techgeeknext</groupId>
	<artifactId>springbootbatch</artifactId>
	<version>0.0.1</version>
	<packaging>jar</packaging>

	<name>springbootbatch</name>
	<description>Spring Boot+Spring Batch+MySQL Example</description>

	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.2.1.RELEASE</version>
		<relativePath /> <!-- lookup parent from repository -->
	</parent>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
		<java.version>1.8</java.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-batch</artifactId>
		</dependency>

		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
			<scope>runtime</scope>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>


</project>

Application Properties

Define database connection details.

## Spring DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.url = jdbc:mysql://localhost:3306/batchExampleDB?createDatabaseIfNotExist=true&allowPublicKeyRetrieval=true&useSSL=false
spring.datasource.username = root
spring.datasource.password = root
spring.datasource.platform=mysql
spring.datasource.initialization-mode=always

## Hibernate Properties
# The SQL dialect makes Hibernate generate better SQL for the chosen database
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL8Dialect

# Hibernate ddl auto (create, create-drop, validate, update)
spring.jpa.hibernate.ddl-auto = create-drop

spring.batch.job.enabled=false

spring.batch.initialize-schema=ALWAYS

spring.batch.job.enabled=false

Add spring.batch.job.enabled to false in application.properties to stop all jobs in the context from being executed by default when the application starts up.

spring.batch.initialize-schema=ALWAYS

Initialize a Spring Batch Database: Setting spring.batch.initialize-schema to ALWAYS value by default create the tables if you are using an embedded database. For more information can visit Spring Batch official documentation.
If you don't add this entry in application.properties, it will throw below exception.

INFO 15304 --- [nio-8080-exec-1] c.t.s.c.BatchJobLauncherController
: PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID,JOB_NAME
from BATCH_JOB_INSTANCE where JOB_NAME = ? and JOB_KEY = ?];
nested exception is java.sql.SQLSyntaxErrorException: Table 'batchexampledb.batch_job_instance'
doesn't exist

Spring Batch Job Configuration

To configure your job, use @Configuration and @EnableBatchProcessing annotations, which adds several important beans to help jobs and saves you a lot of time.

The simpleJob() method defines the job, and the step1() method defines a single step. Jobs are created from steps, where each step can have a reader, a processor, and a writer.

package com.techgeeknext.springbootbatch.config;

import com.techgeeknext.springbootbatch.step.MessageProcessor;
import com.techgeeknext.springbootbatch.step.MessageReader;
import com.techgeeknext.springbootbatch.step.MessageWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class SpringBootBatchConfig {

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    /**
     * A Job is made up of many steps and each step is
     *  a READ-PROCESS-WRITE task or a single operation task (tasklet).
      * @return job
     */
    @Bean
    public Job simpleJob() {
        return jobBuilderFactory.get("simpleJob")
                .incrementer(new RunIdIncrementer())
                .flow(step1())
                .end()
                .build();
    }

    /**
     * Step consist of an ItemReader, ItemProcessor and an ItemWriter.
     * @return step
     */
    @Bean
    public Step step1() {
        return stepBuilderFactory.get("step1")
                .<String, String> chunk(1)
                .reader(new MessageReader())
                .processor(new MessageProcessor())
                .writer(new MessageWriter())
                .build();
    }

}

incrementer(new RunIdIncrementer())

You can notice in code, we have used incrementer. Since jobs uses a database to maintain execution state, you'll need an incrementer in this job definition. After that, you create a list of each step (though this job has only one step). When the job is completed, the Java API generates a perfectly configured job.

In Step1 method, we have defined reader, processor and writer to execute.

Read Data With ItemReader

Create MessageReader Class by implementing ItemReader interface and override read method to read the data.

package com.techgeeknext.springbootbatch.step;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.item.ItemReader;

/**
 * ItemReader
 */
public class MessageReader implements ItemReader<String> {

    private String[] welcomeMessage = {"Hello World!", "Welcome to TechGeekNext Spring Batch Example!"};

    private int msgIndex = 0;

    Logger logger = LoggerFactory.getLogger(this.getClass());

    /**
     * It read the data from the given source
     *
     * @return String
     * @throws Exception
     */
    @Override
    public String read() throws Exception {

        //read and pass message to processor to process the message
        if (msgIndex < welcomeMessage.length) {
            //welcomeMessage[0],welcomeMessage[1]
            return welcomeMessage[msgIndex++];
        } else {
            msgIndex = 0;
        }
        return null;
    }

}

Process Data With ItemProcessor

Create MessageProcessor class by implementing ItemProcessor interface and override process method to write our business logic.

package com.techgeeknext.springbootbatch.step;

import org.springframework.batch.item.ItemProcessor;

/**
 * ItemProcessor
 */
public class MessageProcessor implements ItemProcessor<String, String> {

    /**
     * Read input data from itemReader, and then ItemProcessor applies the business logic here
     *
     * @param content
     * @return String
     * @throws Exception
     */
    @Override
    public String process(String content) throws Exception {
        return "TEST - [" + content.toUpperCase() + "]";
    }

}

Write Data to Destination With ItemWriter

Crate MessageWriter class, implement ItemWriter interface and override write method to write received data to destination.

package com.techgeeknext.springbootbatch.step;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.item.ItemWriter;

import java.util.List;

/**
 * ItemWriter
 */
public class MessageWriter implements ItemWriter<String> {

    Logger logger = LoggerFactory.getLogger(this.getClass());

    /**
     * ItemWriter writes received data to destination.
     *
     * @param inputMessage
     * @throws Exception
     */
    @Override
    public void write(List<? extends String> inputMessage) throws Exception {
        //write data to console
        for (String outputMsg : inputMessage) {
            System.out.println("Received input data from Step:- " + outputMsg);
        }
    }

}

Job Launcher

Create RestApi, which will call job launcher to start the job.

package com.techgeeknext.springbootbatch.controller;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class BatchJobLauncherController {

    @Autowired
    JobLauncher jobLauncher;

    @Autowired
    Job simpleJob;

    /**
     * Method to launch the job
     *
     * @return String
     * @throws Exception
     */
    @RequestMapping("/launch/welcome/job")
    public String jobLauncher() throws Exception {

        Logger logger = LoggerFactory.getLogger(this.getClass());
        try {
            JobParameters jobParameters = new JobParametersBuilder()
                    .addLong("time", System.currentTimeMillis())
                    .toJobParameters();
            //job launcher is an interface for running the jobs
            jobLauncher.run(simpleJob, jobParameters);
        } catch (Exception e) {
            logger.info(e.getMessage());
        }

        return "Job Launched Successfully!";
    }
}

Test

1. Use the http://localhost:8080/launch/welcome/job url, this will start the Job.

2. Once you hit the job launcher url as above, it starts processing the configured job with steps defined.

  .   ____   _ __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot :: (v2.2.1.RELEASE)

13:52:10.441  INFO 20792 --- [main] c.t.s.SpringBootBatchApplication  : Starting SpringBootBatchApplication with PID 20792 (D:\springbootbatch\target\classes started in D:\springbootbatch)
13:52:10.444  INFO 20792 --- [main] c.t.s.SpringBootBatchApplication  : No active profile set, falling back to default profiles: default
13:52:11.533  INFO 20792 --- [main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
13:52:11.543  INFO 20792 --- [main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
13:52:11.543  INFO 20792 --- [main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.27]
13:52:11.649  INFO 20792 --- [main] o.a.c.c.C.[Tomcat].[localhost].[/]: Initializing Spring embedded WebApplicationContext
13:52:11.649  INFO 20792 --- [main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 1168 ms
13:52:11.909  INFO 20792 --- [main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
13:52:12.008  INFO 20792 --- [main] com.zaxxer.hikari.HikariDataSource: HikariPool-1 - Starting...
13:52:12.137  INFO 20792 --- [main] com.zaxxer.hikari.HikariDataSource: HikariPool-1 - Start completed.
13:52:12.202  INFO 20792 --- [main] o.s.b.c.r.s.JobRepositoryFactoryBean     : No database type set, using meta data indicating: MYSQL
13:52:12.214  INFO 20792 --- [main] o.s.b.c.l.support.SimpleJobLauncher      : No TaskExecutor has been set, defaulting to synchronous executor.
13:52:12.317  INFO 20792 --- [main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
13:52:12.319  INFO 20792 --- [main] c.t.s.SpringBootBatchApplication  : Started SpringBootBatchApplication in 2.203 seconds (JVM running for 2.556)
13:52:39.155  INFO 20792 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]: Initializing Spring DispatcherServlet 'dispatcherServlet'
13:52:39.156  INFO 20792 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
13:52:39.165  INFO 20792 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 9 ms
13:52:39.322  INFO 20792 --- [nio-8080-exec-1] o.s.b.c.l.support.SimpleJobLauncher: Job: [FlowJob: [name=simpleJob]] launched with the following parameters: [{time=1621498959183}]
13:52:39.400  INFO 20792 --- [nio-8080-exec-1] o.s.batch.core.job.SimpleStepHandler: Executing step: [step1]
Received input data from Step:- TEST - [HELLO WORLD!]
Received input data from Step:- TEST - [WELCOME TO TECHGEEKNEXT SPRING BATCH EXAMPLE!]
13:52:39.450  INFO 20792 --- [nio-8080-exec-1] o.s.batch.core.step.AbstractStep: Step: [step1] executed in 50ms
13:52:39.466  INFO 20792 --- [nio-8080-exec-1] o.s.b.c.l.support.SimpleJobLauncher: Job: [FlowJob: [name=simpleJob]] completed with the following parameters: [{time=1621498959183}] and the following status: [COMPLETED] in 108ms

3. By default, Spring Batch creates below meta data tables.

Download Source Code

The full source code for this article can be found on below.
Download it here - Spring Boot + Spring Batch + MySQL Simple Example