弹簧批次容错能力

我正在尝试从 csv 文件导入城市数据，某些数据可能会重复，从而引发冲突错误ERROR: duplicate key value violates unique constraint "city__unique_idx" Detail: Key (country, city_name, latitude, longitude)=(231, Monticello, 30.5450000000000017, -83.8703000000000003) already exists. 2018-03-13 14:34:03.607 ERROR 13275 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step cityStep1 in job importCityJob。我想知道如何忽略此错误并保持作业运行，因为它当前立即退出。

Writer:

@Bean
public ItemWriter<City> cityWriter(EntityManagerFactory factory) {
    JpaItemWriter<City> writer = new JpaItemWriter<>();
    writer.setEntityManagerFactory(factory);
    return writer;
}

这是我的工作方法：

@Bean
public Step cityStep1(ItemWriter<City> writer) {
    return stepBuilderFactory.get("cityStep1").<City, City>chunk(10).reader(cityReader())
            .processor(cityProcessor()).writer(writer).build();
}

@Bean
public Job importHotelJob(JobCompletionNotificationListener listener, Step cityStep) {
    return jobBuilderFactory.get("importCityJob").incrementer(new RunIdIncrementer())
            .listener(listener)
            .next(cityStep1)
            .build();

Thanks.

Edit1:

我申请后faultTolerant()

@Bean
public Step cityStep1(ItemWriter<City> writer) {
    return stepBuilderFactory.get("cityStep1").<City, City>chunk(50).reader(cityReader())
            .processor(cityProcessor()).writer(writer)
            .faultTolerant()
            .skip(ConflictException.class)
            .skip(ConstraintViolationException.class)
            .noRetry(ConflictException.class)
            .noRetry(ConstraintViolationException.class)
            .skipLimit(150)
            .build();

我仍然收到错误：

2018-03-14 15:49:40.047 ERROR 26613 --- [           main] o.h.engine.jdbc.spi.SqlExceptionHelper   : ERROR: duplicate key value violates unique constraint "city__unique_idx"
  Detail: Key (country, city_name, latitude, longitude)=(231, Monticello, 30.5450000000000017, -83.8703000000000003) already exists.
2018-03-14 15:49:40.161 ERROR 26613 --- [           main] o.s.batch.core.step.AbstractStep         : Encountered an error executing step cityStep1 in job importCityJob
org.springframework.retry.ExhaustedRetryException: Retry exhausted after last attempt in recovery path, but exception is not skippable.; nested exception is javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement
    at org.springframework.batch.core.step.item.FaultTolerantChunkProcessor$5.recover(FaultTolerantChunkProcessor.java:405) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
    at org.springframework.retry.support.RetryTemplate.handleRetryExhausted(RetryTemplate.java:512) ~[spring-retry-1.2.1.RELEASE.jar:na]
    at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:351) ~[spring-retry-1.2.1.RELEASE.jar:na]
    at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:211) ~[spring-retry-1.2.1.RELEASE.jar:na]

您需要使您的步骤对特定异常具有容错能力。

@Bean
public Step cityStep1(ItemWriter<City> writer) {
    return stepBuilderFactory.get("cityStep1")
                .<City,City>chunk(10)
                .reader(cityReader())
                .processor(cityProcessor())
                .writer(writer)
                .faultTolerant()
                .skip(YourException.class)
                .noRetry(YourException.class)
                .noRollback(YourException.class)
                .skipLimit(10)
                .build();
}

您尚未列出您的作者或您收到的例外情况，因此您可以将该例外情况代替YourException.

noRetry , noRollBack已被用来改变默认的 Spring Batch 行为，以防跳过，因为 Spring Batch 回滚整个块的事务，然后重新处理项目。如果您同意这种方式，则可以从步骤定义中删除这些部分。