Spring Batch Hello world example using Spring boot

Spring Batch is a framework for batch processing - execution series of jobs. In Spring Batch, A job consist of many steps and each step consist of READ-PROCESS-WRITE task and single operation task (tasklet).

READ-PROCESS-WRITE process means It read data from resources like ( csv, xml and database etc.) , process on the data/item according to requirement and write it to the resources like (csv, xml and database).

Tasklet (single task/operation), It is used to do clean up resources after or before step started or completed.

A job is sequence of steps

---

Spring Batch Example

In this example, we will discuss to create simple spring batch example using spring boot. We will show you how to create a job, step, read a CSV file, process it, write the output in XML file.

Consider the following batch job:

1. Read csv file from folder X, process it and write into other folder Y (READ-PROCESS-WRITE).

Project structure

This is a directory structure of the standard gradle project.

Project dependencies

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'

version = '0.0.1'
sourceCompatibility = 1.8

repositories {
    mavenLocal()
    mavenCentral()
}


dependencies {
    compile 'org.springframework:spring-beans'
    compile 'org.springframework:spring-oxm'
    compileOnly('org.projectlombok:lombok')
    compile('org.springframework.boot:spring-boot-starter-batch')
    testCompile('org.springframework.boot:spring-boot-starter-test')
}
buildscript {
    ext {
        springBootVersion = '2.3.5.RELEASE'
    }
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
    }
}

application.properties file

#empty

Spring Batch Jobs

A CSV file

1001,"213.100",980,"Walking Techie", 29/7/2013
1002,"320.200",1080,"Walking Techie 1", 30/7/2013
1003,"342.197",1200,"Walking Techie 2", 31/7/2013

A Spring batch job, to read a CSV file using FlatFileItemReader, process it using ItemProcessor, and write it using StaxEventItemWriter to a XML file.

package com.walking.techie.hello.jobs;

import com.walking.techie.hello.ReportFieldSetMapper;
import com.walking.techie.hello.model.Report;
import com.walking.techie.hello.processor.CustomItemProcessor;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.mapping.DefaultLineMapper;
import org.springframework.batch.item.file.transform.DelimitedLineTokenizer;
import org.springframework.batch.item.xml.StaxEventItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.FileSystemResource;
import org.springframework.oxm.jaxb.Jaxb2Marshaller;

@Configuration
@EnableBatchProcessing
public class HelloWorldJob {

  @Autowired
  private JobBuilderFactory jobBuilderFactory;
  @Autowired
  private StepBuilderFactory stepBuilderFactory;
  @Autowired
  private ReportFieldSetMapper setMapper;


  @Bean
  public FlatFileItemReader<Report> reader() {
    FlatFileItemReader<Report> reader = new FlatFileItemReader<>();
    reader.setResource(new ClassPathResource("report.csv"));
    reader.setLineMapper(new DefaultLineMapper<Report>() {{
      setLineTokenizer(new DelimitedLineTokenizer() {{
        setNames(new String[]{"id", "sales", "qty", "staffName", "date"});
      }});
      setFieldSetMapper(setMapper);
    }});
    return reader;
  }

  @Bean
  public CustomItemProcessor processor() {
    return new CustomItemProcessor();
  }

  @Bean(destroyMethod = "")
  public StaxEventItemWriter<Report> writer() {
    StaxEventItemWriter<Report> writer = new StaxEventItemWriter<>();
    writer.setResource(new FileSystemResource("file:xml/outputs/report.xml"));
    writer.setMarshaller(marshaller());
    writer.setRootTagName("report");
    return writer;
  }

  @Bean
  public Jaxb2Marshaller marshaller() {
    Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
    marshaller.setClassesToBeBound(Report.class);
    return marshaller;
  }

  @Bean
  public Job helloWorld() {
    return jobBuilderFactory.get("helloWorld").incrementer(new RunIdIncrementer())
        .flow(step1()).end().build();
  }

  @Bean
  public Step step1() {
    return stepBuilderFactory.get("step1").<Report, Report>chunk(10).reader(reader())
        .processor(processor()).writer(writer()).build();
  }
}

Map CSV file values to Report object and write to XML file (using jaxb annotations)

A Java model class

package com.walking.techie.hello.model;


import java.math.BigDecimal;
import java.util.Date;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement(name = "record")
public class Report {

  private int id;
  private BigDecimal sales;
  private int qty;
  private String staffName;
  private Date date;

  @XmlAttribute(name = "id")
  public int getId() {
    return id;
  }

  public void setId(int id) {
    this.id = id;
  }

  @XmlElement(name = "sales")
  public BigDecimal getSales() {
    return sales;
  }

  public void setSales(BigDecimal sales) {
    this.sales = sales;
  }

  @XmlElement(name = "qty")
  public int getQty() {
    return qty;
  }

  public void setQty(int qty) {
    this.qty = qty;
  }

  @XmlElement(name = "staffName")
  public String getStaffName() {
    return staffName;
  }

  public void setStaffName(String staffName) {
    this.staffName = staffName;
  }

  @XmlElement(name = "date")
  public Date getDate() {
    return date;
  }

  public void setDate(Date date) {
    this.date = date;
  }

  @Override
  public String toString() {
    return "Report [id=" + id + ", sales=" + sales
        + ", qty=" + qty + ", staffName=" + staffName + "]";
  }
}

This is the processor file which will process each and every item that is read using FlatFileItemReader

A Processor file

package com.walking.techie.hello.processor;

import com.walking.techie.hello.model.Report;
import org.springframework.batch.item.ItemProcessor;

public class CustomItemProcessor implements ItemProcessor<Report, Report> {

  @Override
  public Report process(Report item) throws Exception {
    System.out.println("Processing..." + item);
    return item;
  }
}

To convert a Date, you need a custom FieldSetMapper. If no data type conversion, just use BeanWrapperFieldSetMapper to map the values by name automatically.

package com.walking.techie.hello;

import com.walking.techie.hello.model.Report;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import org.springframework.batch.item.file.mapping.FieldSetMapper;
import org.springframework.batch.item.file.transform.FieldSet;
import org.springframework.stereotype.Component;
import org.springframework.validation.BindException;

@Component
public class ReportFieldSetMapper implements FieldSetMapper<Report> {

  @Override
  public Report mapFieldSet(FieldSet fieldSet) throws BindException {
    SimpleDateFormat sdf = new SimpleDateFormat("dd/MM/yyyy");
    Report report = new Report();
    report.setId(fieldSet.readInt(0));
    report.setSales(fieldSet.readBigDecimal(1));
    report.setQty(fieldSet.readInt(2));
    report.setStaffName(fieldSet.readString(3));
    String date = fieldSet.readString(4);
    try {
      report.setDate(sdf.parse(date));
    } catch (ParseException e) {
      e.printStackTrace();
    }
    return report;
  }
}

Run Application

package com.walking.techie;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;

@SpringBootApplication(exclude = DataSourceAutoConfiguration.class)
public class HelloApplication {

  public static void main(String[] args) {
    SpringApplication.run(HelloApplication.class, args);
  }
}

Output

Output of the application will store in file:xml/outputs/report.xml

<?xml version="1.0" encoding="UTF-8"?>
 <report>
  <record id="1001">
    <date>2013-07-29T00:00:00+05:30</date>
    <qty>980</qty>
    <sales>213.100</sales>
    <staffName>Walking Techie</staffName>
  </record>
  <record id="1002">
    <date>2013-07-30T00:00:00+05:30</date>
    <qty>1080</qty>
    <sales>320.200</sales>
    <staffName>Walking Techie 1</staffName>
  </record>
  <record id="1003">
    <date>2013-07-31T00:00:00+05:30</date>
    <qty>1200</qty>
    <sales>342.197</sales>
    <staffName>Walking Techie 2</staffName>
  </record>
</report>

output on console

2017-03-26 10:53:52.379  INFO 6489 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [FlowJob: [name=helloWorld]] launched with the following parameters: [{run.id=1}]
2017-03-26 10:53:52.397  INFO 6489 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [step1]
Processing...Report [id=1001, sales=213.100, qty=980, staffName=Walking Techie]
Processing...Report [id=1002, sales=320.200, qty=1080, staffName=Walking Techie 1]
Processing...Report [id=1003, sales=342.197, qty=1200, staffName=Walking Techie 2]
2017-03-26 10:53:52.442  INFO 6489 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [FlowJob: [name=helloWorld]] completed with the following parameters: [{run.id=1}] and the following status: [COMPLETED]

Note : This code has been compiled and run on mac notebook and intellij IDEA.

You can find the above working code from git.