Spring Batch Introduction

Spring Batch Introduction

• 1.
  Spring Batch IntroductionTadaya Tsuyukubo twitter: @ttddyy   http://bit.ly/ttddyy_spring-batch-intro
• 2.
  Agenda background architecturedemo more concept summary
• 3.
  Batch Process? bulkprocess long running process mostly sequential onetime, daily, monthly, yearly, ... "The lack of a standard, reusable batch architecture has resulted in the proliferation of many one-off, in-house solutions developed within client enterprise IT functions." - spring batch documentation
• 4.
  Spring Batch Accenture+ SpringSource     Accenture : industry knowledge & experience     SpringSource : Tech, Spring programming model   Batch Process Infrastructure transaction management, skip, repeat, job execution, etc. POJO based not scheduler    
• 5.
  why not hadoopor other grid frameworks? Difference: framework vs runtime environment - no need to have special environment - flexibility of storage and algorithm - reuse of existing java library embedded to existing app HDFS, HBase, MapReduce ... standalone process (possible to distribute) efficient to distribute process Light Weight Heavy Weight (doesn't mean good/bad)
• 6.
  Architecture decision, caseby case   mix both:  pre-process job to hadoop.  ex: transform log and push to hdfs. (flume?)     Guideline:   data & computation small to medium : spring batch very large : hadoop or grid frameworks   * "spring batch in action [MEAP] - manning"
• 7.
  Basic Architecture ItemReader:retrieve input data from datasource(file, database, queue, etc.)   ItemProcessor: transform input data to output data   ItemWriter: persist output data to data store
• 8.
  Chunk Oriented Processinginterface ItemReader<T> {     T read(); }   interface ItemWriter<T> {     void write(List<? extends T> items); }   API (pseudo)
• 9.
  ItemReaders, ItemWriters ItemReader: FlatFileItemReader JdbcCursorItemReader JdbcPagingItemReader HibernateCursorItemReader IbatisPagingItemReader JmsItemReader etc. DataSource Flat File XML Database Message etc. ItemWriter: FlatFileItemWriter HibernateItemWriter JdbcBatchItemWriter JpaItemWriter StaxEventItemWriter etc.
• 10.
  Demo [Source] http://github.com/ttddyy/demo     - spring-batch-intro [Samples] SimpleApp reader, processor, writer FlatfileApp read from csv, passing parameter, late binding, step scope
• 11.
  Step Step:chunk, tasklet Step Flow: sequential conditional
• 12.
  Job - groupof steps - represent entire batch process
• 13.
  Job (cont.) Jobexecution * from spring-batch documentation
• 14.
  JobLauncher, JobRepository JobLaunchersimple API to run job TaskExecutor Synchronous Asynchronous JobRepository store job status/result database, in-memory * from spring-batch documentation Figure2.1: Batch Stereotypes
• 15.
  more topics... parameterpassing (JobParmeters, ExecutionContext) &quot;step&quot; scope chunk processing commit interval, repeat policy, skip policy   listeners ItemReadListener, ItemProcessListener, ItemWriteListener StepExecutionListener, ChunkListener, SkipListener scaling multi-thread, parallel, remote, partitioning   web admin
• 16.
  Summary Spring Batchprovides infrastructure focus on business logic pojo programming with DI   Light weight easily embedded to existing application reuse existing libraries Simple, easy, and powerful    
• 17.
  Reference Spring BatchProject : http://static.springsource.org/spring-batch/ Documentation : http://static.springsource.org/spring-batch/reference/html/index.html Spring Batch In Action [MEAP] ISBN: 9781935182955 http://www.manning.com/templier/