· Added by nasarat anjum, last edited by Arun Varadarajan on Sep 26, 2009  (view change)

Business Context

        In BW infrastructure, data targets are built and there can be a scenario in future to enhance/modify the existing data target . Suppose a DSO is built and we need to add a Key Figure to the DSO. Data would be present in the DSO since a long time and you can't take the pain to load data again into the Same DSO just to populate values into the new Key Figure.

       To face such scenario, an ABAP program can be written which will load data just to that Key Figure corresponding to the old values.

       We faced a similar scenario, where we had a Key figure which was of type "Interger". Now we had to Change this Key Figure to type "Number- FLTP". Changing the existing Key Figure and then Re-loading the existing (10 million records of data) DSO would require lot of time.

The solution we implemented was
1. Create a new Key Figure and include it in the DSO.
2. Create an ABAP program to populate the New Key Figure with the values from the already existing Key Figure.
3. Once this is done for existing data, we can stop loading the old key Figure (by modifying the transformations) and continue loading the New Key figure through the normal data loading procedure.

Advantage(s):

This will save us lot of time needed for re-loading of DSO.

Assumption(s):
Data is loaded into the data target prior to scheduling this program.

Limitation(s):
This program is hard coded to Key Figures, it can be enhanced to make the program generalized to accept Old Key Figure and New Key Figure at run time and do the processing based on the input values.

PROGRAM SOURCE CODE:

*&---------------------------------------------------------------------*
*& Report  YPOPULATE_DSO_KF
*&
*&---------------------------------------------------------------------*
*& This program has a selection variable called "Month". Based on the
*& month entered, the program updates the Key figure values into newly
*& added key figure for all the records corresponding to the month.
*&
*&---------------------------------------------------------------------*

REPORT YPOPULATE_DSO_KF.

***SELECTION SCREEN
***--------------------------------------------------------------------*
TABLES: /BIC/AZDSO_O0100,
        /BIC/AZDSO_O0200,
        /BIC/AZDSO_O0300,
        /BIC/AZDSO_O0400.

SELECTION-SCREEN: BEGIN OF BLOCK B1 WITH FRAME TITLE TEXT-001.
PARAMETERS L_DSO(10) TYPE C.
SELECT-OPTIONS: L_EFFMTH FOR /BIC/AZDSO_O0100-/BIC/ZEFF_MNTH.
SELECTION-SCREEN: END OF BLOCK B1.
*************************************************************************

* Check whether Month value is entered or not.
IF L_EFFMTH IS INITIAL.
* To check which DSO is to be updated.
CASE L_DSO.
* If 'ZDSO_O01'is selected.
WHEN 'ZDSO_O01'.
UPDATE /BIC/AZDSO_O0100 SET /BIC/ZNEWFLD = /BIC/AZDSO_O0100~/BIC/ZOLDFLD.
* If 'ZDSO_O02' is selected.
WHEN 'ZDSO_O02'.
UPDATE /BIC/AZDSO_O0200 SET /BIC/ZNEWFLD = /BIC/AZDSO_O0200~/BIC/ZOLDFLD.
* If 'ZDSO_O03'is selected.
WHEN 'ZDSO_O03'.
UPDATE /BIC/AZDSO_O0300 SET /BIC/ZNEWFLD = /BIC/AZDSO_O0300~/BIC/ZOLDFLD.
* If 'ZDSO_O04' is selected.
WHEN 'ZDSO_O04'.
UPDATE /BIC/AZDSO_O0400 SET /BIC/ZNEWFLD = /BIC/AZDSO_O0400~/BIC/ZOLDFLD.
ENDCASE.
* Check whether Effective Month value entered.
ELSE.
CASE L_DSO.
* If 'ZDSO_O01'is selected
WHEN 'ZDSO_O01'.
UPDATE /BIC/AZDSO_O0100  SET /BIC/ZNEWFLD = /BIC/AZDSO_O0100~/BIC/ZOLDFLD WHERE /BIC/ZEFF_MNTH IN L_EFFMTH.
* If 'ZDSO_O02' is selected.
WHEN 'ZDSO_O02'.
UPDATE /BIC/AZDSO_O0200 SET /BIC/ZNEWFLD = /BIC/AZDSO_O0200~/BIC/ZOLDFLD WHERE /BIC/ZEFF_MNTH IN L_EFFMTH.
* If 'ZDSO_O03'is selected.
WHEN 'ZDSO_O03'.
UPDATE /BIC/AZDSO_O0300 SET /BIC/ZNEWFLD = /BIC/AZDSO_O0300~/BIC/ZOLDFLD WHERE /BIC/ZEFF_MNTH IN L_EFFMTH.
* If 'ZDSO_O04' is selected.
WHEN 'ZDSO_O04'.
UPDATE /BIC/AZDSO_O0400 SET /BIC/ZNEWFLD = /BIC/AZDSO_O0400~/BIC/ZOLDFLD WHERE /BIC/ZEFF_MNTH IN L_EFFMTH.
ENDCASE.
ENDIF.

Image may be NSFW.
Clik here to view.

· Added by Krish, last edited by Arun Varadarajan on Mar 16, 2009

InfoCube:
Definition
An object that can function as both a data target and an InfoProvider.

From a reporting point of view, an InfoCube describes a self-contained dataset, for example, of a business-orientated area. This dataset can be evaluated in a BEx query.        

An InfoCube is a quantity of relational tables arranged according to the star schema: A large fact table in the middle surrounded by several dimension tables.

Use
InfoCubes are supplied with data from one or more InfoSources or ODS objects (Basic InfoCube) or with data from a different system (RemoteCube, SAP RemoteCube, virtual InfoCube with Services, transactional InfoCube).

Structure
There are various types of InfoCube:
1.      Physical data stores:

Basic InfoCubes
Transactional InfoCubes

2.      Virtual data stores:
RemoteCube
SAP RemoteCube
Virtual InfoCube with Services

Only Basic InfoCubes and transactional InfoCubes physically contain data in the database. Virtual InfoCubes are only logical views of a dataset. By definition, they are not data targets. However, the InfoCube type is of no importance from the reporting perspective, since an InfoCube is accessed as an InfoProvider.

Integration
You can access the characteristics and key figures defined for an InfoCube in the Query Definition in the BEx Web or in the BEx Analyzer.

Basic InfoCube
Definition
A Basic InfoCube is a type of InfoCube that physically stores data. It is filled with data using BW Staging. Afterwards, it can be used as an InfoProvider in BEx Reporting.
Structure
As with other InfoCube types, the structure of a Basic InfoCube corresponds to the Star Schema.
For more information, see InfoCube
Integration
The Basic InfoCube is filled using the Scheduler, providing that Update Rules have been maintained.
It is then made available to Reporting as an InfoProvider. It can also be updated into additional data targets or build a MultiProvider together with other data targets.

Transactional InfoCubes
Definition
Transactional InfoCubes differ from Basic InfoCubes in their ability to support parallel write accesses. Basic InfoCubes are technically optimized for read accesses to the detriment of write accesses.
Use
Transactional InfoCubes are used in connection with the entry of planning data. See also Overview of Planning with BW-BPS. The data from this kind of InfoCube is accessed transactionally, meaning data is written to the InfoCube (possibly by several users at the same time). Basic InfoCubes are not suitable for this. You should use Basic InfoCubes for read-only access (for example, when reading reference data).
Structure
Transactional InfoCubes can be filled with data using two different methods: Using the transaction of BW-BPS to enter planning data and using BW staging, whereas planning data then cannot be loaded simultaneously. You have the option to convert a transactional Info Cube Select Convert Transactional InfoCube using the context menu in your transactional InfoCube in the InfoProvider tree. By default, Transaction Cube Can Be Planned, Data Loading Not Permitted is selected. Switch this setting to Transactional Cube Can Be Loaded With Data; Planning Not Permitted  if you want to fill the cube with data via BW Staging.

During entry of planning data, the data is written to a transactional InfoCube data request. As soon as the number of records in a data request exceeds a threshold value, the request is closed and a rollup is carried out for this request in defined aggregates (asynchronously).  You can still rollup and define aggregates, collapse, and so on, as before.

According to the database on which they are based, transactional InfoCubes differ from Basic InfoCubes in the way they are indexed and partitioned.  For an Oracle DBMS this means, for example, no bitmap index for the fact table and no partitioning (initiated by SAP BW) of the fact table according to the package dimensions.

Reduced read-only performance is accepted as a drawback of transactional InfoCubes, in the face of the parallel (transactional) writing option and improved write performance.

Creating a transactional InfoCube
Select the Transactional indicator when creating a new (Basis) InfoCube in the Administrator Workbench.
Converting a basic InfoCube into a transactional InfoCube
InfoCube conversion: Removing transaction data
If the Basic InfoCube already contains transaction data that you no longer need (for example, test data from the implementation phase of the system), proceed as follows:
1.      In the InfoCube maintenance in the Administrator Workbench choose, from the main menu, InfoCube ® Delete Data Content. The transaction data is deleted and the InfoCube is set to "inactive".
2.      Continue with the same procedure as with creating a transactional InfoCube.
InfoCube conversion: retaining transaction data
If the Basic InfoCube already contains transaction data from the production operation you still need, proceed as follows:
Execute the SAP_CONVERT_TO_TRANSACTIONAL ABAP report under the name of the corresponding InfoCube. You should schedule this report as a background job for InfoCubes with more than 10,000 data records. This is to avoid a potentially long run-time.
Integration

The following typical scenarios arise for the use of transactional InfoCubes in BW-BPS.
1st Scenario:
Actual data (read-only access) and planned data (read-only and write access) have to be held in different InfoCubes. Therefore, use a Basic InfoCube for actual data and a transactional InfoCube for planned data. Data integration is achieved using a multi-planning area that contains the areas that are assigned to the InfoCubes. Access to the two different InfoCubes is controlled here by the characteristic "Planning area", which is added automatically.
2nd Scenario:
In this scenario, the planned and actual data have to be together in one InfoCube. This is the case, for example, with special rolling forecast variants. Here you have to use a transactional InfoCube, since both read-only and write accesses take place. You can no longer load data directly that has already arrived in the InfoCube by means of an upload or import source. To be able to load data nevertheless, you have to make a copy of the transactional InfoCube that is identified as a Basic InfoCube and not as transactional. Data is loaded as usual here and subsequently updated to the transactional InfoCube.

RemoteCube
Definition
A RemoteCube is an InfoCube whose transaction data is not managed in the Business Information Warehouse but externally. Only the structure of the RemoteCube is defined in BW. The data is read for reporting using a BAPI from another system.
Use
Using a RemoteCube, you can carry out reporting using data in external systems without having to physically store transaction data in BW. You can, for example, include an external system from market data providers using a RemoteCube.
By doing this, you can reduce the administrative work on the BW side and also save memory space.
Structure
When reporting using a RemoteCube, the Data Manager, instead of using a BasicCube filled with data, calls the RemoteCube BAPI and transfers the parameters.
Selection
Characteristics
Key figures
As a result, the external system transfers the requested data to the OLAP Processor.
Integration
To report using a RemoteCube you have to carry out the following steps:
1.      In BW, create a source system for the external system that you want to use.
2.      Define the required InfoObjects.
3.      Load the master data:
Create a master data InfoSource for each characteristic Load texts and attributes
4.      Define the RemoteCube
5.      Define the queries based on the RemoteCube

SAP RemoteCube
Definition
An SAP RemoteCube is a RemoteCube that allows the definition of queries with direct access to transaction data in other SAP systems.
Use
Use SAP RemoteCubes if:
You need very up-to-date data from an SAP source system
You only access a small amount of data from time to time
Only a few users execute queries simultaneously on the database.
Do not use SAP RemoteCubes if:
You request a large amount of data in the first query navigation step, and no appropriate aggregates are available in the source system
A lot of users execute queries simultaneously
You frequently access the same data
Structure
SAP RemoteCubes are defined based on an InfoSource with flexible updating. They copy the characteristics and key figures of the InfoSource. Master data and hierarchies are not read directly in the source system. They are already replicated in BW when you execute a query.
The transaction data is called during execution of a query in the source system. During this process, the selections are provided to the InfoObjects if the transformation is only simple mapping of the InfoObject. If you have specified a constant in the transfer rules, the data is transferred only if this constant can be fulfilled. With more complex transformations such as routines or formulas, the selections cannot be transferred. It takes longer to read the data in the source system because the amount of data is not limited. To prevent this you can create an inversion routine for every transfer routine. Inversion is not possible with formulas, which is why SAP recommends that you use formulas instead of routines.
Integration
To be assigned to an SAP RemoteCube, a source system must meet the following requirements:
BW Service API functions (contained in the SAP R/3 plug-in) are installed.
The Release status of the source system is at least 4.0B
In BW, a source system ID has been created for the source system
DataSources from the source system that are released for direct access are assigned to the InfoSource of the SAP RemoteCube. There are active transfer rules for these combinations.

Virtual InfoCubes with Services
Definition
A virtual InfoCube with services is an InfoCube that does not physically store its own data in BW. The data source is a user-defined function module. You have a number of options for defining the properties of the data source more precisely. Depending on these properties, the data manager provides services to convert the parameters and data.
Use
You use a virtual InfoCube with services if you want to display data from non-BW data sources in BW without having to copy the data set into the BW structures. The data can be local or remote. You can also use your own calculations to change the data before it is passed to the OLAP processor.
This function is used primarily in the SAP Strategic Enterprise Management (SEM) application.
In comparison to the RemoteCube, the virtual InfoCube with services is more generic. It offers more flexibility, but also requires more implementation effort.
Structure
When you create an InfoCube you can specify the type. If you choose Virtual InfoCube with Services as the type for your InfoCube, an extra Detail pushbutton appears on the interface. This pushbutton opens an additional dialog box, in which you define the services.
1.      Enter the name of the function module that you want to use as the data source for the virtual InfoCube. There are different default variants for the interface of this function module. One method for defining the correct variant, together with the description of the interfaces, is given at the end of this documentation.
2.      The next step is to select options for converting/simplifying the selection conditions. You do this by selecting the Convert Restrictions option. These conversions only change the transfer table in the user-defined function module. The result of the query is not changed because the restrictions that are not processed by the function module are checked later in the OLAP processor.
Options:
No restrictions: If this option is selected, no restrictions are passed to the InfoCube.
Only global restrictions: If this option is selected, only global restrictions (FEMS = 0) are passed to the function module. Other restrictions (FEMS > 0) that are created, for example, by setting restrictions on columns in queries, are deleted.
Simplify selections: Currently this option is not yet implemented.
Expand hierarchy restrictions: If this option is selected, restrictions on hierarchy nodes are converted into the corresponding restrictions on the characteristic value.
3.      Pack RFC: This option packs the parameter tables in BAPI format before the function module is called and unpacks the data table that is returned by the function module after the call is performed. Since this option is only useful in conjunction with a remote function call, you have to define a logical system that is used to determine the target system for the remote function call, if you select this option.
4.      SID support: If the data source of the function module can process SIDs, you should select this option.
If this is not possible, the characteristic values are read from the data source and the data manager determines the SIDs dynamically. In this case, wherever possible, restrictions that are applied to SID values are converted automatically into the corresponding restrictions for the characteristic values.
5.      With navigation attributes: If this option is selected, navigation attributes and restrictions applied to navigation attributes are passed to the function module.
If this option is not selected, the navigation attributes are read in the data manager once the user-defined function module has been executed. In this case, in the query, you need to have selected the characteristics that correspond to these attributes. Restrictions applied to the navigation attributes are not passed to the function module in this case.
6.      Internal format (key figures): In SAP systems a separate format is often used to display currency key figures. The value in this internal format is different from the correct value in that the decimal places are shifted. You use the currency tables to determine the correct value for this internal representation.
If this option is selected, the OLAP processor incorporates this conversion during the calculation.
Dependencies
If you use a remote function call, SID support must be switched off and the hierarchy restrictions must be expanded.
Description of the interfaces for user-defined function modules
Variant 1: Variant 2:
Additional parameters for variant 2 for transferring hierarchy restrictions, if they are not expanded:
With hierarchy restrictions, an entry for the 'COMPOP' = 'HI' (for hierarchy) field is created at the appropriate place in table I_T_RANGE (for FEMS 0) or I_TX_RANGETAB (for FEMS > 0), and the 'LOW' field contains a number that can be used to read the corresponding hierarchy restriction from table I_TSX_HIER, using field 'POSIT' .  Table i_tsx_hier has the following type:
Variant 3:
SAP advises against using this interface.
The interface is intended for internal use only and only half of it is given here.
Note that SAP may change the structures used in the interface.
Method for determining the correct variant for the interface
The following list describes the procedure for determining the correct interface for the user-defined function module. Go through the list from top to the bottom. The first appropriate case is the variant that you should use:
If Pack RFC is activated: Variant 1
If SID Support is deactivated: Variant 2

Image may be NSFW.
Clik here to view.

· Added by B. Rajesh, last edited by Arun Varadarajan.

Hi,

A small description about DSO types in BI is as follows, just for your reference.

Image may be NSFW.
Clik here to view.

DataStore object types:

Standard DataStore object

· Data provided using a data transfer process

· SID values can be generated

· Data records with the same key are aggregated during activation

· Data is available for reporting after activation

Write-optimized DataStore object

· Data provided using a data transfer process

· SID values cannot be generated

· Records with the same key are not aggregated

· Data is available for reporting immediately after it is loaded

DataStore object for direct update

· Data provided using APIs

· SIDs cannot be generated

· Records with the same key are not aggregated

And you can find more information / examples on this topic at:

http://help.sap.com/saphelp_nw04s/helpdata/en/F9/45503C242B4A67E10000000A114084/content.htm

Thanks and Regrds

Image may be NSFW.
Clik here to view.

· Added by Vincent Judge, last edited by Arun Varadarajan on Jul 28, 2009

ROOSPRMSC 
RLOGSYS: target system 
SLOGSYS: source system 
INITRNR: init request 
DELTARNR: last delta request 
INITSTATE: if X than the delta update is active

ROOSPRMSF selection criteria for init request
FIELDNM 
RSLOW 
RSHIGH

RSSDLINIT
LOGSYS: source system
RNR: Init request number
INITSTATE: = X -> Delta update is active
LAST DELTA RNR: last delta update

RSSDLINITSELselection criteria for init request
IOBJNM
FIELDNAME
RSLOW
RSHIGH
RSSDLINITDELall deleted initalisations

Image may be NSFW.
Clik here to view.

· Added by Davide Cavallari, last edited by Arun Varadarajan on Mar 09, 2010  (view change)

Image may be NSFW.
Clik here to view.

• Attachments:11
• Added by Ashok Babu. - ON Demand Business Team, last edited by Arun Varadarajan on Mar 17, 2009  (view change)

Data Transfer Process and Error handling process
Introduction about Data Transfer Process

1. You use the data transfer process (DTP) to transfer data within BI from a persistent object to another object in accordance with certain transformations and filters. In this respect, it replaces the data mart interface and the Info Package. As of SAP Net Weaver 2004s, the Info Package only loads data to the entry layer of BI (PSA).
2. The data transfer process makes the transfer processes in the data warehousing layer more transparent. Optimized parallel processing improves the performance of the transfer process .You can use the data transfer process to separate delta processes for different targets and you can use filter options between the persistent objects on various levels. For example, you can use filters between a Data Store object and an Info Cube.
3. Data transfer processes are used for standard data transfer, for real-time data acquisition and for accessing data directly. 
   Image may be NSFW.
   Clik here to view.
   

Interesting Benefits of New Data Transfer Process

1. Loading data from one layer to others except Info sources.
2. Separation of delta mechanism for different data targets.
3. Enhanced filtering in dataflow.
4. Improved transparency of staging processes across data warehouse layers.
5. Improved performance : optimized parallelization
6. Enhanced error handling in the form of error stack
7. Enables real-time data acquisition. 
   Image may be NSFW.
   Clik here to view.

Most important advantage in Data Transfer Process

1. Delta logic can be separately handled for separate data targets
2. Example for separation for delta logic
3. Delta logic is a part of DTP
4. One Source PSA
5. Two targets : One DSO keeping daily data and other one keeping weekly data

Five process for handling errors in DTP

Process#1 Enhanced Filtering, Debugging and error handling options
Image may be NSFW.
Clik here to view.

Process # 2 -Handling Data Records With Errors

1. Using the error handling settings on the Update tab page in the data transfer process, when data is transferred from a DTP source to a DTP target, you can specify how the system is to react if errors occur in the data records.
2. These settings were previously made in the Info Package. When using data transfer processes, Info Packages write to the PSA only. Error handling settings are therefore no longer made in the Info Package, but in the data transfer process

Process # 3 -Error Handling Features

1. Possibility to choose in the scheduler to
2. Abort process when errors occur
3. Process the correct records but do not allow reporting on them
4. Process the correct records and allow reporting on them
5. Number of wrong records which lead to a wrong request
6. Invalid records can be written into an error stack
7. Keys should be defined for error stack to enable the error handling of data store object
8. Temporary data storage can be switched on/off for each sub step of the loading process
9. Invalid records can be updated into data records after their correction

Image may be NSFW.
Clik here to view.

Process # 4 - Error Stack

1. Stores erroneous records
2. Keeps the right sequence of records à for consistent data store handling.
3. Key of error stack defines which data should be detained from the update after the erroneous data record.
4. After Correction, Error-DTP updates data from error stack to data target.

Note: Once the request in the source object is deleted, the related data records in error stack area automatically deleted.

1. 1. Key of Error Stack = Semantic Groups.
   2. Subset of the key of the target object.
      1. Max. 16 fields
      2. Defines which data should be detained from the update of erroneous data record (for data store object)
      3. The bigger the key, the fewer records will be written to the error stack Image may be NSFW.
         Clik here to view.
         Process # 5 - Temporary Data Storage
2. In order to analyze the data at various stages you can activate the temporary storage in the DTP
3. This allows you to determine the reasons of error Image may be NSFW.
   Clik here to view.

    I enjoyed sharing my knowledge. Hope the above information is useful and provide me your feed back. Thank you.

And related link as follow: http://help.sap.com/saphelp_nw2004s/helpdata/en/42/f98e07cc483255e10000000a1553f7/frameset.htm

Image may be NSFW.
Clik here to view.

• Added by Mahesh Kumar, last edited by Arun Varadarajan

Once older verion (3.x) queries are migrated into latest version (7.0).Query can be restored to 3.x after migration. The backup for any query created in 3.x will be taken when first opened for editing in 7.0 Query Designer. The backup contains the last change done to the query using 3.x editor, any changes done in 7.0 Query Designer will be lost on restore. Also the query originally created in 7.0 can not be restored to older versions as there is no Backup in 3.x.

   Queries can be restored to 3.x version using program COMPONENT_RESTORE.

Steps followed for Restoring Query with 3.x versions:

Step 1 : Execute "COMPONENT_RESTORE" program in SE38.

Image may be NSFW.
Clik here to view.
Step 2 :   Next screen Select InfoProvider and component type. Different component types are available like

Step 3 : Select REP as a component type to revert back Query.  
Image may be NSFW.
Clik here to view.
Image may be NSFW.
Clik here to view.
Step 4 :  Execute (F8). In the next screen it will displays all related Queries for that particular infoprovider.
Image may be NSFW.
Clik here to view.

Step 5 : Search for the query which you want to revert back to older versions.

Step 6 : Then say transfer selection. below message will appears in the system.
Image may be NSFW.
Clik here to view.

Step 7 : Once we select Yes, Query successfully restored into older versions.

Image may be NSFW.
Clik here to view.

• Added by Arun Varadarajan, last edited by Arun Varadarajan

Image may be NSFW.
Clik here to view.

Raj

Image may be NSFW.
Clik here to view. Long time ago I was sitting back thinking about the architecture of ODS and how invaluble it is from its inception with BW 2.0 in the BW Data Warehouse Layer (DWH). This basically motivated me to write this blog with the necessary tech details of ODS - Well call it Data Store Object (DSO) as of NW04s or BI 7.0

"An Operational Data Store object (ODS object) is used to store consolidated and cleansed data (transaction data or master data for example) on a document level (atomic level)" - Refered from SAP Docs.It describes a consolidated dataset from one or more Info Sources / transformations (7.0) as illustrated below in Fig.1.
In this blog we will look at the Standard Data Store Object. We have other types namely Data Store Object with Direct Update (Transactional ODS in 3.x) and Write Optimized Data Store new with BI 7.x which contains only Active data table used to manage huge data loads for instance - Here is the link from Help portal Write optimised DSO

Architecture of Standard ODS /DSO (7.x)
"ODS Objects consist of three tables as shown in the Architecture graphic below" - Refered from SAP Docs:

Image may be NSFW.
Clik here to view.
Figure 1: ODS Architecture - Extracted from SAP Docs

TIP: The new data status is written to the table with active data in parallel to writing to the change log taking the advantage of parallel processes which can be customized globally or at the object level in the system

Lets go through a Scenario
In this example we will take the Master data object material and plant (0MAT_PLANT compounded with 0PLANT) with a few attributes for the demonstration purpose. Now define a ODS / DSO as below where material and plant is a key and the corresponding attributes as data fields.

Image may be NSFW.
Clik here to view.
Figure 2: ODS / DSO definition

Lets create a flat file data source or an info source with 3.x in this example to simplify the scenario with all the info objects we have defined in ODS structure

Image may be NSFW.
Clik here to view.
Figure 3: Info source definition
Lets check the flat file records, remember that the key fields are plant and material and we have a duplicate record as in the below Fig.4. The 'Unique Data Records'option is unchecked which means it can expect duplicate records.
Image may be NSFW.
Clik here to view.
Figure 4: Flat file Records
Check the monitor entries and we see that 3 records are transferred to update rules and two records are loaded in to NEWDATA table as we haven't activated the request yet. This is because we have a duplicate record for the key in the ODS which gets overwritten (Check the first two records in Fig 4)
Image may be NSFW.
Clik here to view.
Figure 5: Monitor Entries
Now check the data in the NEWDATA / ACTIVATION QUEUE table, we have only two records as the duplicate records gets overwritten with the most recent record i.e. record 2 in PSA got overwritten as it has got the same key material and plant.
Image may be NSFW.
Clik here to view.
Figure 6: Activation Queue
Image may be NSFW.
Clik here to view.
Figure 7: PSA data for comparison
Tip: The key figures will have the overwrite option by default, additionally we have the summation option to suit certain scenarios and the characteristics will overwrite always. The technical name of new data / Activation queue table is always for customer objects - /bic <name of ODS>140 and for SAP objects - /bio<name of ODS>140.
Once we activate the data we will have two records in ODS Active Data table. As we see below the Active Data table always has contains the semantic key (Material, Plant)
Image may be NSFW.
Clik here to view.
Figure 8: Active Data Table
TIP: The name of the active table /BIC/A<odsname>100 and /BI0 for SAP.
The change log table has these 2 entries with the new image (N). Remember the record mode we will look in to it later. The technical key (REQID, DATAPACKETID, RECORD NUMBER) will be part of change log
Image may be NSFW.
Clik here to view.
Figure 9: Change Log Table
TIP: The technical name is always /BIC/<internal generated number>.
Now we will add two new records material 75 plant 1, Material 80 Plant 1 and change the existing record for the key Material 1 and plant 1 as below
Image may be NSFW.
Clik here to view.
Figure 10: Add more records
When we look at the monitor there will be 3 records in the activation queue table as the duplicate records gets filtered out, in this example the first record in Fig.10
Image may be NSFW.
Clik here to view.
Figure 11: Monitor
Look at the new data table (Activation Queue) and we will have 3 records that are updated as seen in the monitor
Image may be NSFW.
Clik here to view.
Figure 12: Activation Queue
How the Change log works?
We will check the change log table to see how the deltas are handled. The highlighted records are from the first request that is uniquely identified by technical key (Request Number, Data packet number, Partition value of PSA and Data record number)
Image may be NSFW.
Clik here to view.
Figure 13: Change log Table 1

With the second load i.e. the second request the change log table puts the before and after Image for the relevant records (the non highlighted part from the Fig.13)

In the above example Material (1) and Plant (1) has the before image with record mode "x"(row 3 in the above Fig) And all the key figures will be have the "-" sign as we have opted to overwrite option and the characteristics will be overwritten always.
Image may be NSFW.
Clik here to view.
Figure 14: Change log Table 2
The after image " " which reflects the change in the data record (Check row 4 in the above fig). We have changed the characteristic Profit center with SE from SECOND and the Key figure Processing Time is changed from 1 to 2. A new record (last row in the above Fig) is added is with the Status "N" as it's a new record.

Summary
This gives us an overview of the standard ODS object and how the change log works. The various record modes available:
Image may be NSFW.
Clik here to view.
Figure 15: Record Modes
Check the note 399739 about the details of the Record Mode. The record mode(s) that a particular data source uses for the delta mechanism largely depends on the type of the extractor. Check the table RODELTM about the BW Delta Process methods with the record modes available as well our well known table ROOSOURCE for the extractor specific delta method.
For Instance LO Cockpit extractors use 'ABR' delta method that supplies After-Image, Before-Image, New Image and Reverse Image. Extractors in HR and Activity based costing uses the delta method 'ADD' i.e. with record mode 'A' and FI-GL,AR,AP extractors are based on delta method 'AIE' i.e. record mode space ' ' After image. The list goes on ..........

Raj   is a SAP certified NW BI Consultant

Image may be NSFW.
Clik here to view.

Added by MAYURI SINHA

Data Extractor Enhancement - Overview of Approach
Following are the steps for data source enhancements:

The flowchart explains the step we need to follow while enhancing a standard LIS extractor.
Determine the fields with which the extractor is to enhanced. Recheck if these fields are not the ones which are already provided by SAP. As if they are already provided, we need not enhance them. As a next check we need to check the LBWE pool where we have some extra fields which we can easily add to our datasource.
If these fields do not exist in the LBWE pool either, we need to enhance the data source.
Above flowchart would give an overview of the same, but here we would go through each of those steps in detail. The approach which we are going to discuss here is the Function Module approach of data source enhancement.
Steps of Data Source Enhancement -Function Module approach
Step 1: Go to T Code CMOD and choose the project you are working on.
Step 2: Choose the exit which is called when the data is extracted. 
Step 3: This is the step where we have a difference from the normal approach.

Normal Approach: CMOD Code

Code Sample:

    WHEN '2LIS_05_Q0ACTY'.

-------
Note: This is just a sample code. Logic will vary according to the requirements.In this normal approach, which is followed in most of the BW instances, we write ABAP CASE/WHEN conditions.
Function ModuleApproach: CMOD Code
*&---------------------------------------------------------------------*
*&  Include           ZXRSAU01                                         *
*&---------------------------------------------------------------------*
DATA: L_FNAME  TYPE RS38L_FNAM,
      L_EXP_FNAME type rs38l_fnam,
      L_EXP_ACTIVE TYPE RS38L_GLOB,
      L_ACTIVE TYPE RS38L_GLOB,
      L_S_SELECT TYPE RSSELECT.
SELECT SINGLE FUNC
         INTO L_FNAME

         FROM ZTEST

        WHERE DSNAM  = I_DATASOURCE.
*&---------------------------------------------------------------------*
* Cehck to see if a local versio of the extractor exists
*&---------------------------------------------------------------------*
IF L_FNAME IS NOT INITIAL.
  CALL FUNCTION 'RS_FUNCTION_ACTIVE_CHECK'
    EXPORTING
      FUNCNAME  = L_FNAME
    IMPORTING
      ACTIVE    = L_ACTIVE
    EXCEPTIONS
      NOT_FOUND = 1
      OTHERS    = 2.
  IF SY-SUBRC EQ 0 AND L_ACTIVE IS NOT INITIAL.
    CALL FUNCTION L_FNAME
      EXPORTING
        I_DATASOURCE             = I_DATASOURCE
        I_ISOURCE                = I_ISOURCE
        I_UPDMODE                = I_UPDMODE
      TABLES
        I_T_SELECT               = I_T_SELECT
        I_T_FIELDS               = I_T_FIELDS
        C_T_DATA                 = C_T_DATA
        C_T_MESSAGES             = C_T_MESSAGES
      EXCEPTIONS
        RSAP_CUSTOMER_EXIT_ERROR = 1.
  ENDIF.                               " IF SY-SUBRC EQ 0...
ELSE.
  CLEAR L_FNAME.
  CONCATENATE 'ZTEST_' I_DATASOURCE INTO L_FNAME.
  CALL FUNCTION 'RS_FUNCTION_ACTIVE_CHECK'
    EXPORTING
      FUNCNAME  = L_FNAME
    IMPORTING
      ACTIVE    = L_ACTIVE
    EXCEPTIONS
      NOT_FOUND = 1
      OTHERS    = 2.
  IF SY-SUBRC = 0 AND L_ACTIVE = 'X'.
    CALL FUNCTION L_FNAME
      EXPORTING
        I_DATASOURCE             = I_DATASOURCE
        I_ISOURCE                = I_ISOURCE
        I_UPDMODE                = I_UPDMODE
      TABLES
        I_T_SELECT               = I_T_SELECT
        I_T_FIELDS               = I_T_FIELDS
        C_T_DATA                 = C_T_DATA
        C_T_MESSAGES             = C_T_MESSAGES
      EXCEPTIONS
        RSAP_CUSTOMER_EXIT_ERROR = 1.
  ENDIF.                               " IF SY-SUBRC = 0...
ENDIF.                                 " IF L_FNAME IS NOT INITIAL.
Note: This is a reusable code. Here ZTEST is a table which maintains the data source name and the function module corresponding to that data source.

Step 4: Here in this step we create a function module for each data source. We create a new FM (Function Module in SE37)
Data Extractor Enhancement - Best Practice/Benefits
  This is the best practice of data source enhancement. This has the following benefits:

1.       No more locking of CMOD code by 1 developer stopping others to enhance other extractors.

2.       Testing of an extractor becomes more independent than others.

3.       Faster and a more robust Approach

Image may be NSFW.
Clik here to view.

Added By: Venkata Chalapathi Challapalli, R. Prem Kumar, Vikas Agarwal, Seema John and Tapan Kumar Jain.

1. Introduction
As we can read from http://help.sap.com SAP Business Information Warehouse provided preconfigured objects under collective term “Business Content” (BC). Business Content includes DataSources, InfoObjects, data targets, and InfoSources that support the entire flow of data within BW. It provides a rapid starting point when modeling key business information requirements, and it is also intended to cover most of the traditional reporting requirements that companies face. However, Business Content will not solve all your data information needs.

Now the spontaneous question is: what’s happen if a standard Business Content DataSource as provided in standard (ready-to-use) configuration doesn’t completely meet our data model requirements?

The Answer is : DataSource Enhancement

Below links shows the ways to do the Datasource Enhancement.

SAP BW Data Source Enhancement

Data Source Enhancement using User Exit

Enhancing LO Datasource Step by Step

Data Extraction & DS Enhancement in SAP BI - Step by Step

Image may be NSFW.
Clik here to view.

Added From Sven van Leuken’s Blog

The great thing about an infoset is that it can combine data from different sources via a "JOIN" (inner/outer) mechanism. This may come in very handy if you want to report on two different sources (apples & oranges) without having to show the annoying '#'.
But what to do if you want to show data from the Apples bucket, even though this bucket is empty for the corresponding Oranges bucket? (and vice versa)
Eg.
Apples: 10 pieces in January 2010 for customer A
Oranges: 5 pieces in January 2010 for customer B
With an infoset (JOIN) on Apples & Oranges (where Apples is 'leading') it is impossible to show the Oranges of January for customer B because Apples has no customer B entry.
To be able to show the Oranges data, you have to use a multiprovider which uses the UNION mechanism. The downside of the multiprovider is that it will show a # for January Oranges for Customer A and a # for January Apples for Customer B.
This unwanted # behaviour can be overruled by using constant selection
Step 1: Create restricted keyfigure
Image may be NSFW.
Clik here to view.
Step 2: Tick on constant selection
Image may be NSFW.
Clik here to view.
Without constant selection:
January 2010;Customer A; Apples 10; Oranges #
January 2010;Customer B; Apples #; Oranges 5
With constant selection:
January 2010;Customer A; Apples 10; Oranges 5
January 2010;Customer B; Apples 10; Oranges 5

Image may be NSFW.
Clik here to view.

Added From Arun KK Blog sapbikk

Writing a Virtual characteristic is one which i really struggled to get on - more because of lack of material than the complexity itself.
Here, i have documented my approach to write one - hope it is a good reference for some one planning to write one.
CAUTION: Virtual char and KFs seriously impact the performance of the query; therefore use them with discretion.
---------------------------------------

Need for virtual char & KFs:

To do calculation/manipulation of char or KFs at query run-time

Walk-through:

We’ll go through the virtual char & KFs using this example.

‘Calc Date’ is an IO that holds the below logic

If Current Date > Revised Promise Date Then

Calc Date = ‘PD’

Else

Calc Date = Revised Promise Date

We shall see how this is implemented using virtual characteristics.

Step 1: Creation of dummy IO

Create a ‘dummy’ info object (without any mapping) that would be the holder for Calc Date

I created IO - ZCRPDTCLC.

Step 2: Associating IO to data target

Add this info object to the data target that would be used for reporting.

I added ZCRPDTCLC under DSO ZPP_DS06 and again to ZPU_M03 MP.

The next steps would involve writing the code for calculating Calc Date.

The code is written in 3 modules.

ZXRSRTOP - Global declaration of variables is done here

ZXRSRU02 - Mode of access for each of the IOs used in the exit is defined here.

ZXRSRZZZ - Association of the global variable to the actual IO is done here.

The exit logic is also written here.

Step 3: Global declaration of variables

Go to ZXRSRTOP module in ABAP editor (tcode se38)

In the ‘Include for Virtual Char’ block (it could be in other blocks also, code written here to make it more organized), declare global variables for Calc Date and Revised Promise Date (these are the objects that will be used in the calculation)

The global variables declared should be in the format –

g_pos__

Data type should be number.

Eg:

Data:

g_pos_ZPP_DS06_ZCRPDTCLC TYPE I,

g_pos_ZPP_DS06_ZCRPDT TYPE I.

Step 4: Defining mode of access for the IO.

Go to ZXRSRU02 module in ABAP editor (tcode se38)

There will be a single CASE block for structure i_s-rkb1d

Eg:

CASE i_s_rkb1d-infocube.

…

ENDCASE

This structure ‘i_s_rkb1d’ has all details regarding the query like query tech name, data target, etc.

Thereby, i_s_rkb1d-infocube will contain the data target for each query.

CASE i_s_rkb1d-infocube.

WHEN 'ZPP_DS06'.

* Fields to read

g_t_chanm-mode = rrke_c_mode-read.

g_t_chanm = 'ZCRPDT'.

APPEND g_t_chanm to e_t_chanm.

* Fields to write

g_t_chanm-mode = rrke_c_mode-no_selection.

g_t_chanm-chanm = 'ZCRPDTCLC'.

APPEND g_t_chanm to e_t_chanm.

ENDCASE.

We check the info cube attribute for the corresponding data target related to our query.

‘rrke_c_mode’ is the structure that defines the mode of access for each IO (read mode, write mode).

‘g_t_chanm’ is the structure that will hold the name of characteristics that will be used

‘g_t_kyfnm’ is the structure that will hold the name of key figures that will be used

In our example, we use only characteristics and hence only ‘g_t_kyfnm’ structure.

The rest of the code should be self-explanatory.

For each new object, you assign its tech name to ‘g_t_chanm_chanm’ object and append it to ‘e_t_chanm’ which is the output structure.

Similarly for key figures, ‘e_t_kyfnm’ is the output structure.However for key figures, there is no structure to set the mode of access (mode of access – read/write by default).

Another example to drive the point home:

* Characteristics and Units

g_t_chanm-mode = rrke_c_mode-read.

g_t_chanm-chanm = '0MATERIAL'. append g_t_chanm to e_t_chanm.

g_t_chanm-chanm = '0PLANT'. append g_t_chanm to e_t_chanm.

g_t_chanm-mode = rrke_c_mode-no_selection.

g_t_chanm-chanm = 'PR_ID1'. append g_t_chanm to e_t_chanm.

g_t_chanm-chanm = 'PR_ID2'. append g_t_chanm to e_t_chanm.

g_t_chanm-chanm = 'PR_YEAR1'. append g_t_chanm to e_t_chanm.

g_t_chanm-chanm = 'PR_YEAR2'. append g_t_chanm to e_t_chanm.

g_t_chanm-chanm = 'PR_CURR1'. append g_t_chanm to e_t_chanm.

g_t_chanm-chanm = 'PR_CURR2'. append g_t_chanm to e_t_chanm.

* Key Figures

append '0QUANT_B' to e_t_kyfnm.

append 'AMOUNT1' to e_t_kyfnm.

append 'AMOUNT2' to e_t_kyfnm.

For ‘g_t_kyfnm’ we need not set any mode.

Step 5: Writing the logic for virtual char/KF

Go to ZXRSRZZZ module in ABAP editor (tcode se38)

Here, create a new form for the data target being used.

Form name should be begin with ‘USER_’ followed by the data target’s name.

C_S_DATE is the structure that would hold the data in the query.

To access the IOs in the code, we have to create field symbols that act as aliases.

Then the global variables created are associated to these aliases using the ASSIGN statement.

The rest of the code involves the implementation logic as in the snippet below.

FORM USER_ZPP_DS06 USING I_S_RKB1D TYPE RSR_S_RKB1D

CHANGING C_S_DATE TYPE ANY.

DATA: L_DATE TYPE SCAL-DATE.

DATA: L_WEEK TYPE SCAL-WEEK.

CONSTANTS: PAST_DUE(2) TYPE C VALUE 'PD'.

FIELD-SYMBOLS: , .

ASSIGN COMPONENT g_pos_ZPP_DS06_ZCRPDT OF STRUCTURE C_S_DATE TO .

ASSIGN COMPONENT g_pos_ZPP_DS06_ZCRPDTCLC OF STRUCTURE C_S_DATE TO .

L_DATE = SY-DATUM. "Today's Date

CALL FUNCTION 'DATE_GET_WEEK'

EXPORTING

DATE = L_DATE

IMPORTING

WEEK = L_WEEK.

IF L_WEEK GT . "If Current Week is greater than Revised Promise Date, Calc_Date holds "PD"

= PAST_DUE.

ELSE. "Calc_Date holds Revised Promise Date

= .

ENDIF.

ENDFORM.

Overall Program Flow for Calc Date

ZXRSRTOP

Data:

g_pos_ZPP_DS06_ZCRPDTCLC TYPE I,

g_pos_ZPP_DS06_ZCRPDT TYPE I.

ZXRSRU02

CASE i_s_rkb1d-infocube.

WHEN 'ZPP_DS06'.

* Fields to read

g_t_chanm-mode = rrke_c_mode-read.

g_t_chanm = 'ZCRPDT'.

APPEND g_t_chanm to e_t_chanm.

* Fields to write

g_t_chanm-mode = rrke_c_mode-no_selection.

g_t_chanm-chanm = 'ZCRPDTCLC'.

APPEND g_t_chanm to e_t_chanm.

ENDCASE.

ZXRSRZZZ

FORM USER_ZPP_DS06 USING I_S_RKB1D TYPE RSR_S_RKB1D

CHANGING C_S_DATE TYPE ANY.

DATA: L_DATE TYPE SCAL-DATE.

DATA: L_WEEK TYPE SCAL-WEEK.

CONSTANTS: PAST_DUE(2) TYPE C VALUE 'PD'.

FIELD-SYMBOLS: , .

ASSIGN COMPONENT g_pos_ZPP_DS06_ZCRPDT OF STRUCTURE C_S_DATE TO .

ASSIGN COMPONENT g_pos_ZPP_DS06_ZCRPDTCLC OF STRUCTURE C_S_DATE TO .

L_DATE = SY-DATUM. "Today's Date

CALL FUNCTION 'DATE_GET_WEEK'

EXPORTING

DATE = L_DATE

IMPORTING

WEEK = L_WEEK.

IF L_WEEK GT . "If Current Week is greater than Revised Promise Date, Calc_Date holds "PD"

= PAST_DUE.

ELSE. "Calc_Date holds Revised Promise Date

= .

ENDIF.

ENDFORM.

Image may be NSFW.
Clik here to view.

Stages in BW project
1 Project Preparation / Requirement Gathering
2 Business Blueprint
3 Realization
4 Final Preparation
5 GO Live & Support

1. Project Preparation / Requirement Gathering
Collect requirement thru interviews with Business teams /Core users / Information Leaders .
Study & analyze KPI 's (key figures) of Business process .
Identify the measurement criteria's (Characteristics).
Understand the Drill down requirements if any.
Understand the Business process data flow if any .
Identify the needs for data staging layers in BW – (i.e need for ODS if any)Understand the system landscape .
Prepare Final Requirements Documents in the form of Functional  specifications containing :
Report Owners,
Data flow ,
KPI’s ,
measurement criteria’s,
Report format along with drilldown requirements .

2. Business Blueprint
Check Business content against the requirements
Check for appropriate
Info Objects - Key figures & Characters
Check for Info cubes / ODS
Check for data sources & identify fields in source system
Identify Master data
document all the information in a file – follow standard templates
Prepare final solution
Identify differences (Gaps) between Business Content & Functional
specification. propose new solutions/Developments & changes if required at different levels such as Info Objects ,Info cube , Data source etc . Document the gaps & respective solutions proposed– follow standard templates
Design & Documentation
Design the ERD & MDM diagrams for each cube & related objects
Design the primary keys/data fields for intermediate Storage in ODS
Design the Data flow charts right from data source up to Cube .
Consider the performance parameters while designing data models
Prepare High level / Low level design documents for each data model.--- follow standard templates
Identify the Roles & Authorizations required and Document it – follow standard templates
final review of design with core BW users .
Sign off the BBP documents

3. Realization
Check & Apply Latest Patches/Packages ...in BW & R/3 systems.
Activate/Build & enhance the cubes/ODS as per data model designs...maintain the version documents .
Identify & activate Info objects / Master data info sources / attributes ,prepare update rules
Assign data sources .prepare transfer rules , prepare multi providers . prepare Info packages .
perform the unit testing for data loads….both for master data & transaction data .
develop & test the end user queries .
Design the process chains ,schedule & test
create authorizations / Roles …assign to users ..and test
Apply necessary patches & Notes if any .
freeze & release the final objects to quality systems
perform quality tests .
Re design if required . (document changes, maintain versions)

4. Final Preparation
Prepare the final check list of objects to be released .identify the dependencies & sequence of release
perform Go Live checks as recommended by SAP in production system
keep up to date Patch Levels in Production system
Test for production scenarios in a pre-production system which is a replica of production system .
Do not Encourage the changes at this stage .
freeze the objects .

5 GO Live & Support
keep up to date Patch Levels
Release the objects to production system
Run the set ups in R/3 source system & Initialize Loads in BW
Schedule Batch jobs in R/3 system (Delta loads)
schedule the process chains in BW .
Performance tuning – on going activity
Enhancements - if any

Image may be NSFW.
Clik here to view.