Saturday, 2 May 2015

Difference between Delta Load and Incremental Load

Scenario:
We have completed the first load into the target table, and wish to implement the Delta load and the Incremental load.

  • Delta Data is the parent of Incremental Data.
  • What is Delta Data?
    • Assuming that in our scenario, the first load happened on 1-Jan-2015. The data present in the source on 2-Jan -2015 is the delta data. 
  • How to read delta data?
    • Add the date filter in the source query i.e. WHERE load_date>1-Jan-2015
  • What is incremental data?
    • The data that we have read , as delta data, needs to be compared with the target data for key columns. This is generally done through SCD(Slowly Changing Dimensions) algorithms. Incremental data is the changed or new data found while the delta and target data are compared via SCD algorithms.






No comments:

Post a Comment