0
Ich habe einen Frühling-Batch-Job geschrieben, der aus einer DB liest und dann auf eine CSV schreiben.Spring Batch ItemWriter Erstellen von Ausgabe zweimal
Der Job ist ziemlich einfach und es funktioniert, aber der ItemWriter sieht so aus, als ob er zweimal ausgeführt wird. d. h. ich lande mit zwei CSVs (genau das gleiche) anstelle von einem.
Kann mir jemand helfen, herauszufinden, warum bitte? Dies geschieht mit einer anderen jobExecution.id.
Anbringen Batch-Job und Konsolenausgabe von unten laufen ...
@Configuration
@EnableBatchProcessing
public class ExportMasterListCsvJobConfig {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Bean
public Job exportMasterListCsvJob(
Step readStgDbAndExportMasterListStep,
Step writeToCsvStep) {
return jobBuilderFactory.get("exportMasterListCsvJob")
.incrementer(new RunIdIncrementer())
.flow(readStgDbAndExportMasterListStep)
.end()
.build();
}
@Bean
public Step readStgDbAndExportMasterListStep(
@Value("${exportMasterListCsv.generateMasterListRows.chunkSize}") int chunkSize,
@Qualifier("queryOnlineStagingDbReader") ItemReader<MasterList> queryOnlineStagingDbReader,
@Qualifier("masterListFileWriter") ItemWriter<MasterList> masterListFileWriter) {
return stepBuilderFactory.get("readStgDbAndExportMasterListStep")
.<MasterList,MasterList>chunk(chunkSize)
.reader(queryOnlineStagingDbReader)
.writer(masterListFileWriter)
.build();
}
@Bean
public ItemReader<MasterList> queryOnlineStagingDbReader(
DataSource onlineStagingDb,
@Value("${exportMasterListCsv.generateMasterListRows.masterListSql}") String masterListSql) {
JdbcCursorItemReader<MasterList> reader = new JdbcCursorItemReader<>();
reader.setDataSource(onlineStagingDb);
reader.setSql(masterListSql);
reader.setRowMapper(new RowMapper<MasterList>() {
@Override
public MasterList mapRow(ResultSet resultSet, int i) throws SQLException {
MasterList masterList = new MasterList();
masterList.setL2(resultSet.getString("l2"));
masterList.setL2Name(resultSet.getString("l2_name"));
return masterList;
}
});
return reader;
}
@Bean
@StepScope
public ItemWriter<MasterList> masterListFileWriter(
FileSystemResource masterListFile,
@Value("#{stepExecutionContext}")Map<String, Object> executionContext) {
FlatFileItemWriter<MasterList> writer = new FlatFileItemWriter<>();
writer.setResource(masterListFile);
DelimitedLineAggregator<MasterList> lineAggregator = new DelimitedLineAggregator<>();
lineAggregator.setDelimiter(",");
BeanWrapperFieldExtractor<MasterList> extractor = new BeanWrapperFieldExtractor<MasterList>();
extractor.setNames(new String[] { "l2", "l2Name"});
lineAggregator.setFieldExtractor(extractor);
writer.setLineAggregator(lineAggregator);
writer.setForceSync(true);
writer.open(new ExecutionContext(executionContext));
return writer;
}
@Bean
@JobScope
public FileSystemResource masterListFile(
@Value("${temp.dir}") String tempDir,
@Value("#{jobExecution.id}") Long jobId
) throws IOException {
return new FileSystemResource(new File(tempDir, "masterList" + jobId + ".csv"));
}
}
und Konsolenausgabe ...
13:03:45.815 [main] DEBUG org.springframework.test.context.junit4.SpringJUnit4ClassRunner - SpringJUnit4ClassRunner constructor called with [class com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.827 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating CacheAwareContextLoaderDelegate from class [org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate]
13:03:45.836 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating BootstrapContext using constructor [public org.springframework.test.context.support.DefaultBootstrapContext(java.lang.Class,org.springframework.test.context.CacheAwareContextLoaderDelegate)]
13:03:45.853 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating TestContextBootstrapper for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest] from class [org.springframework.test.context.support.DefaultTestContextBootstrapper]
13:03:45.884 [main] DEBUG org.springframework.test.context.support.DefaultTestContextBootstrapper - Found explicit ContextLoader class [org.springframework.boot.test.SpringApplicationContextLoader] for context configuration attributes [[email protected] declaringClass = 'com.myer.pricing.onlinestore.export.ExportMasterListCsvTest', classes = '{class com.myer.pricing.onlinestore.export.TestJobConfig, class com.myer.pricing.onlinestore.export.job.ExportMasterListCsvJobConfig}', locations = '{}', inheritLocations = true, initializers = '{}', inheritInitializers = true, name = [null], contextLoaderClass = 'org.springframework.boot.test.SpringApplicationContextLoader']
13:03:45.899 [main] DEBUG org.springframework.test.context.support.AbstractContextLoader - Did not detect default resource location for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]: class path resource [com/myer/pricing/onlinestore/export/ExportMasterListCsvTest-context.xml] does not exist
13:03:45.900 [main] DEBUG org.springframework.test.context.support.AbstractContextLoader - Did not detect default resource location for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]: class path resource [com/myer/pricing/onlinestore/export/ExportMasterListCsvTestContext.groovy] does not exist
13:03:45.900 [main] INFO org.springframework.test.context.support.AbstractContextLoader - Could not detect default resource locations for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]: no resource found for suffixes {-context.xml, Context.groovy}.
13:03:45.927 [main] INFO org.springframework.test.context.support.DefaultTestContextBootstrapper - Using TestExecutionListeners: [[email protected]724af044, org.springframework.test[email protected]4678c730, org.springframewor[email protected]6767c1fc, org.springfra[email protected]29ee9faa, org.springframew[email protected]c038203, org.sp[email protected]cc285f4]
13:03:45.931 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved @ProfileValueSourceConfiguration [null] for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.931 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved ProfileValueSource type [class org.springframework.test.annotation.SystemProfileValueSource] for class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.946 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved @ProfileValueSourceConfiguration [null] for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.946 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved ProfileValueSource type [class org.springframework.test.annotation.SystemProfileValueSource] for class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.948 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved @ProfileValueSourceConfiguration [null] for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.948 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved ProfileValueSource type [class org.springframework.test.annotation.SystemProfileValueSource] for class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.950 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved @ProfileValueSourceConfiguration [null] for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.950 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved ProfileValueSource type [class org.springframework.test.annotation.SystemProfileValueSource] for class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.953 [main] DEBUG org.springframework.test.context.support.AbstractDirtiesContextTestExecutionListener - Before test class: context [[email protected] testClass = ExportMasterListCsvTest, testInstance = [null], testMethod = [null], testException = [null], mergedContextConfiguration = [[email protected] testClass = ExportMasterListCsvTest, locations = '{}', classes = '{class com.myer.pricing.onlinestore.export.TestJobConfig, class com.myer.pricing.onlinestore.export.job.ExportMasterListCsvJobConfig}', contextInitializerClasses = '[]', activeProfiles = '{batchtest}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextLoader = 'org.springframework.boot.test.SpringApplicationContextLoader', parent = [null]]], class annotated with @DirtiesContext [false] with mode [null].
13:03:45.954 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved @ProfileValueSourceConfiguration [null] for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.954 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved ProfileValueSource type [class org.springframework.test.annotation.SystemProfileValueSource] for class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.959 [main] DEBUG org.springframework.test.util.ReflectionTestUtils - Getting field 'mergedContextConfiguration' from target object [[[email protected] testClass = ExportMasterListCsvTest, testInstance = [email protected]2d7, testMethod = [null], testException = [null], mergedContextConfiguration = [[email protected] testClass = ExportMasterListCsvTest, locations = '{}', classes = '{class com.myer.pricing.onlinestore.export.TestJobConfig, class com.myer.pricing.onlinestore.export.job.ExportMasterListCsvJobConfig}', contextInitializerClasses = '[]', activeProfiles = '{batchtest}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextLoader = 'org.springframework.boot.test.SpringApplicationContextLoader', parent = [null]]]] or target class [class org.springframework.test.context.support.DefaultTestContext]
13:03:45.960 [main] DEBUG org.springframework.test.util.ReflectionTestUtils - Setting field 'propertySourceProperties' of type [null] on target object [[[email protected] testClass = ExportMasterListCsvTest, locations = '{}', classes = '{class com.myer.pricing.onlinestore.export.TestJobConfig, class com.myer.pricing.onlinestore.export.job.ExportMasterListCsvJobConfig}', contextInitializerClasses = '[]', activeProfiles = '{batchtest}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextLoader = 'org.springframework.boot.test.SpringApplicationContextLoader', parent = [null]]] or target class [class org.springframework.test.context.MergedContextConfiguration] to value [[Ljava.lang.String;@4667ae56]
13:03:45.961 [main] DEBUG org.springframework.test.context.support.DependencyInjectionTestExecutionListener - Performing dependency injection for test context [[[email protected] testClass = ExportMasterListCsvTest, testInstance = [email protected]2d7, testMethod = [null], testException = [null], mergedContextConfiguration = [[email protected] testClass = ExportMasterListCsvTest, locations = '{}', classes = '{class com.myer.pricing.onlinestore.export.TestJobConfig, class com.myer.pricing.onlinestore.export.job.ExportMasterListCsvJobConfig}', contextInitializerClasses = '[]', activeProfiles = '{batchtest}', propertySourceLocations = '{}', propertySourceProperties = '{org.springframework.boot.test.IntegrationTest=true}', contextLoader = 'org.springframework.boot.test.SpringApplicationContextLoader', parent = [null]]]].
13:03:46.072 [main] DEBUG org.springframework.core.env.StandardEnvironment - Adding [systemProperties] PropertySource with lowest search precedence
13:03:46.073 [main] DEBUG org.springframework.core.env.StandardEnvironment - Adding [systemEnvironment] PropertySource with lowest search precedence
13:03:46.074 [main] DEBUG org.springframework.core.env.StandardEnvironment - Initialized StandardEnvironment with PropertySources [systemProperties,systemEnvironment]
13:03:46.074 [main] DEBUG org.springframework.core.env.StandardEnvironment - Adding [test] PropertySource with highest search precedence
13:03:46.075 [main] DEBUG org.springframework.core.env.StandardEnvironment - Adding [integrationTest] PropertySource with search precedence immediately lower than [systemEnvironment]
. ____ _ __ _ _
/\\/___'_ __ _ _(_)_ __ __ _ \ \ \ \
(()\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ))))
' |____| .__|_| |_|_| |_\__, |////
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.3.6.RELEASE)
2016-07-22 13:03:46.541 INFO 10084 --- [ main] c.m.p.o.export.ExportMasterListCsvTest : Starting ExportMasterListCsvTest on IGSM473537 with PID 10084 (C:\design_centre_dev\myer-pricing-engine\java-onlinestore-feed-exporter\target\test-classes started by rriviere in C:\design_centre_dev\myer-pricing-engine\java-onlinestore-feed-exporter)
2016-07-22 13:03:46.542 INFO 10084 --- [ main] c.m.p.o.export.ExportMasterListCsvTest : The following profiles are active: batchtest
2016-07-22 13:03:46.875 INFO 10084 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.spring[email protected]4e08711f: startup date [Fri Jul 22 13:03:46 AEST 2016]; root of context hierarchy
2016-07-22 13:03:48.151 INFO 10084 --- [ main] o.s.b.f.config.PropertiesFactoryBean : Loading properties file from URL [jar:file:/C:/.m2/repository/org/springframework/integration/spring-integration-core/4.2.8.RELEASE/spring-integration-core-4.2.8.RELEASE.jar!/META-INF/spring.integration.default.properties]
2016-07-22 13:03:48.157 INFO 10084 --- [ main] o.s.i.config.IntegrationRegistrar : No bean named 'integrationHeaderChannelRegistry' has been explicitly defined. Therefore, a default DefaultHeaderChannelRegistry will be created.
2016-07-22 13:03:48.716 WARN 10084 --- [ main] o.s.c.a.ConfigurationClassEnhancer : @Bean method ScopeConfiguration.stepScope is non-static and returns an object assignable to Spring's BeanFactoryPostProcessor interface. This will result in a failure to process annotations such as @Autowired, @Resource and @PostConstruct within the method's declaring @Configuration class. Add the 'static' modifier to this method to avoid these container lifecycle issues; see @Bean javadoc for complete details.
2016-07-22 13:03:48.728 WARN 10084 --- [ main] o.s.c.a.ConfigurationClassEnhancer : @Bean method ScopeConfiguration.jobScope is non-static and returns an object assignable to Spring's BeanFactoryPostProcessor interface. This will result in a failure to process annotations such as @Autowired, @Resource and @PostConstruct within the method's declaring @Configuration class. Add the 'static' modifier to this method to avoid these container lifecycle issues; see @Bean javadoc for complete details.
2016-07-22 13:03:48.742 INFO 10084 --- [ main] faultConfiguringBeanFactoryPostProcessor : No bean named 'errorChannel' has been explicitly defined. Therefore, a default PublishSubscribeChannel will be created.
2016-07-22 13:03:48.751 INFO 10084 --- [ main] faultConfiguringBeanFactoryPostProcessor : No bean named 'taskScheduler' has been explicitly defined. Therefore, a default ThreadPoolTaskScheduler will be created.
2016-07-22 13:03:48.900 INFO 10084 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [class org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$402d705e] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2016-07-22 13:03:49.060 INFO 10084 --- [ main] o.s.j.d.e.EmbeddedDatabaseFactory : Starting embedded database: url='jdbc:hsqldb:mem:testdb', username='sa'
2016-07-22 13:03:49.459 INFO 10084 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executing SQL script from class path resource [db/sql/staging_db.sql]
2016-07-22 13:03:49.465 INFO 10084 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executed SQL script from class path resource [db/sql/staging_db.sql] in 6 ms.
2016-07-22 13:03:49.710 WARN 10084 --- [ main] o.s.b.c.l.AbstractListenerFactoryBean : org.springframework.batch.item.ItemWriter is an interface. The implementing class will not be queried for annotation based listener configurations. If using @StepScope on a @Bean method, be sure to return the implementing class so listner annotations can be used.
2016-07-22 13:03:49.797 INFO 10084 --- [ main] o.s.aop.framework.CglibAopProxy : Unable to proxy method [public final java.lang.String org.springframework.core.io.FileSystemResource.getPath()] because it is final: All calls to this method via a proxy will NOT be routed to the target instance.
2016-07-22 13:03:50.236 INFO 10084 --- [ main] o.s.b.f.config.PropertiesFactoryBean : Loading properties file from URL [jar:file:/C:/.m2/repository/org/springframework/integration/spring-integration-core/4.2.8.RELEASE/spring-integration-core-4.2.8.RELEASE.jar!/META-INF/spring.integration.default.properties]
2016-07-22 13:03:50.730 INFO 10084 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService 'taskScheduler'
2016-07-22 13:03:51.258 INFO 10084 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executing SQL script from class path resource [org/springframework/batch/core/schema-hsqldb.sql]
2016-07-22 13:03:51.281 INFO 10084 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executed SQL script from class path resource [org/springframework/batch/core/schema-hsqldb.sql] in 23 ms.
2016-07-22 13:03:51.813 INFO 10084 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase 0
2016-07-22 13:03:51.814 INFO 10084 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2016-07-22 13:03:51.814 INFO 10084 --- [ main] o.s.i.channel.PublishSubscribeChannel : Channel 'application:batchtest.errorChannel' has 1 subscriber(s).
2016-07-22 13:03:51.814 INFO 10084 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started _org.springframework.integration.errorLogger
2016-07-22 13:03:51.832 INFO 10084 --- [ main] o.s.b.a.b.JobLauncherCommandLineRunner : Running default command line with: []
2016-07-22 13:03:51.844 INFO 10084 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: HSQL
2016-07-22 13:03:51.973 INFO 10084 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2016-07-22 13:03:52.044 INFO 10084 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=exportMasterListCsvJob]] launched with the following parameters: [{run.id=1}]
2016-07-22 13:03:52.073 INFO 10084 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [readStgDbAndExportMasterListStep]
2016-07-22 13:03:52.167 INFO 10084 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=exportMasterListCsvJob]] completed with the following parameters: [{run.id=1}] and the following status: [COMPLETED]
2016-07-22 13:03:52.169 INFO 10084 --- [ main] c.m.p.o.export.ExportMasterListCsvTest : Started ExportMasterListCsvTest in 6.091 seconds (JVM running for 6.895)
2016-07-22 13:03:52.196 INFO 10084 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=exportMasterListCsvJob]] launched with the following parameters: [{random=93853}]
2016-07-22 13:03:52.234 INFO 10084 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [readStgDbAndExportMasterListStep]
2016-07-22 13:03:52.290 INFO 10084 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=exportMasterListCsvJob]] completed with the following parameters: [{random=93853}] and the following status: [COMPLETED]
2016-07-22 13:03:52.305 INFO 10084 --- [ Thread-1] s.c.a.AnnotationConfigApplicationContext : Closing org.spring[email protected]4e08711f: startup date [Fri Jul 22 13:03:46 AEST 2016]; root of context hierarchy
2016-07-22 13:03:52.307 INFO 10084 --- [ Thread-1] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 0
2016-07-22 13:03:52.308 INFO 10084 --- [ Thread-1] o.s.i.endpoint.EventDrivenConsumer : Removing {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2016-07-22 13:03:52.309 INFO 10084 --- [ Thread-1] o.s.i.channel.PublishSubscribeChannel : Channel 'application:batchtest.errorChannel' has 0 subscriber(s).
2016-07-22 13:03:52.309 INFO 10084 --- [ Thread-1] o.s.i.endpoint.EventDrivenConsumer : stopped _org.springframework.integration.errorLogger
2016-07-22 13:03:52.314 INFO 10084 --- [ Thread-1] o.s.s.c.ThreadPoolTaskScheduler : Shutting down ExecutorService 'taskScheduler'
2016-07-22 13:03:52.317 INFO 10084 --- [ Thread-1] o.s.j.d.e.EmbeddedDatabaseFactory : Shutting down embedded database: url='jdbc:hsqldb:mem:testdb'
Und das erzeugt zwei CSV die jeweils mit einem anderen jobExecution.id die ich habe an den Dateinamen angehängt. masterList0.csv UND masterList1.csv
Dank im Voraus