Implementing Batch Processing
1. Adding Spring Batch Dependencies (spring-boot-starter-batch)
Example: Spring Batch dependency
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
| Auto-configured |
Bean |
| JobRepository |
Persists job/step state |
| JobLauncher |
Launches jobs |
| JobExplorer |
Read-only job metadata |
| BatchProperties |
spring.batch.* |
2. Enabling Batch Processing (@EnableBatchProcessing)
Note: In Spring Boot 3+, batch is auto-enabled when starter is on classpath.
Use @EnableBatchProcessing only to override the default configuration.
Example: Batch application entry point
@SpringBootApplication
public class BatchApp { public static void main(String[] a) { SpringApplication.run(BatchApp.class, a); } }
3. Creating Job Configuration (Job, Step)
Example: Job and chunk-oriented step
@Configuration
public class ImportJobConfig {
@Bean
Job importUsers(JobRepository jr, Step step) {
return new JobBuilder("importUsers", jr).start(step).build();
}
@Bean
Step step(JobRepository jr, PlatformTransactionManager tx,
ItemReader<User> r, ItemProcessor<User,User> p, ItemWriter<User> w) {
return new StepBuilder("step", jr)
.<User,User>chunk(100, tx).reader(r).processor(p).writer(w).build();
}
}
| Reader |
Source |
FlatFileItemReader |
CSV/TXT |
JdbcCursorItemReader |
JDBC cursor |
JdbcPagingItemReader |
Paged JDBC |
JpaPagingItemReader |
JPA |
StaxEventItemReader |
XML |
JsonItemReader |
JSON |
Example: Normalize and filter items in processor
@Bean
ItemProcessor<User, User> normalizeProcessor() {
return user -> {
user.setEmail(user.getEmail().toLowerCase().trim());
return user.getEmail().contains("@") ? user : null; // null filters out
};
}
6. Implementing ItemWriter for Data Output
Example: Batch writer using JPA repository
@Bean
ItemWriter<User> writer(UserRepository repo) {
return chunk -> repo.saveAll(chunk.getItems());
}
7. Configuring Chunk-Oriented Processing
| Chunk Size |
Trade-off |
| Small (10-50) |
Lower memory, more commits |
| Medium (100-1000) |
Balanced (typical) |
| Large (10k+) |
Higher throughput, risk of OOM/lock |
8. Using FlatFileItemReader for CSV/TXT
Example: Read CSV with FlatFileItemReader
@Bean
FlatFileItemReader<User> csvReader() {
return new FlatFileItemReaderBuilder<User>()
.name("userReader")
.resource(new ClassPathResource("users.csv"))
.linesToSkip(1)
.delimited().names("id","name","email")
.targetType(User.class)
.build();
}
9. Using JdbcBatchItemWriter for Database
Example: Batch JDBC writer with named parameters
@Bean
JdbcBatchItemWriter<User> jdbcWriter(DataSource ds) {
return new JdbcBatchItemWriterBuilder<User>()
.dataSource(ds)
.sql("INSERT INTO users (id,name,email) VALUES (:id,:name,:email)")
.beanMapped()
.build();
}
10. Implementing Job Parameters (JobParameters)
Example: Build and run job with parameters
JobParameters params = new JobParametersBuilder()
.addLocalDate("runDate", LocalDate.now())
.addString("inputFile", "users-2024-01.csv")
.toJobParameters();
jobLauncher.run(importUsers, params);
11. Scheduling Batch Jobs (@Scheduled)
Example: Schedule nightly batch job
@Component
public class BatchScheduler {
private final JobLauncher launcher;
private final Job importUsers;
public BatchScheduler(JobLauncher l, Job j) { this.launcher = l; this.importUsers = j; }
@Scheduled(cron = "0 0 2 * * *")
public void nightly() throws Exception {
launcher.run(importUsers, new JobParametersBuilder()
.addLong("ts", System.currentTimeMillis()).toJobParameters());
}
}
12. Monitoring Batch Job Execution (JobRepository)
| API |
Use |
JobExplorer.getJobInstances(name, start, count) |
List instances |
JobExplorer.getJobExecutions(instance) |
Per-instance executions |
JobOperator.restart(execId) |
Restart failed |
Actuator /actuator/batch |
Custom endpoint exposure |
13. Implementing Skip and Retry Logic
Example: Fault-tolerant step with skip and retry
return new StepBuilder("step", jr)
.<User,User>chunk(100, tx)
.reader(r).processor(p).writer(w)
.faultTolerant()
.skip(FlatFileParseException.class).skipLimit(50)
.retry(TransientException.class).retryLimit(3)
.build();