Scan:
Scan generates an output record for each input record that includes a running cumulative
summary for the group to which the input record belongs.
Properties of Scan:
Sorted-input: There are 2 values for this parameter. When set to “Input must be sorted or
grouped”, the component requires grouped inputs which are sorted. This is the default option
for this parameter. When set to “In-memory: Input need not be sorted”, the component
accepts ungrouped input and it requires use of max_core parameter.
Max-core: Maximum memory usage in bytes. Default is 10MB.
Key-method: Method by which component group records. There are 2 options for this
parameter.
Use Key Specifier
Use Key_change function
Key: Name of the key fields, that the component will use to group records are mentioned here.
This parameter is available only when Key method parameter is set as “Use Key specifier”.
Check-sort: If you use a key specifier, this parameter indicates whether or not to abort
execution on the first input record that is out of sorted order.
Transform: The name of the file containing the types and transform functions, or a transform
string.
Reject-threshold: The component's tolerance for reject events.
Logging: Specifies whether or not to log component events.
Scan works for parallel files also without serializing them it will consider each
partition as different file and does the processing.
Sample Graph
Example 1
To Assign sequence Number:
Input
record
decimal("|") cust_id;
string("|") cust_name;
decimal("|") bill_no;
decimal("\n") amount;
end
Code in Transform
type temporary_type=
record
decimal(4) index;
end; /*Temporary variable*/
temp :: initialize(in) =
begin
temp.index :: 0;
end;
temp :: scan(temp, in) =
begin
temp.index :: temp.index+1;
end;
out :: finalize(temp, in) =
begin
out.Sno :: temp.index;
out.* :: in.*;
end;
Output
record
decimal("|") Sno;
decimal("|") cust_id;
string("|") cust_name;
decimal("|") bill_no;
decimal("\n") amount;
end
*******************************************************************
Example 2
To show key_change function:
Input
record
decimal("|") cust_id;
string("|") cust_name;
decimal("|") bill_no;
decimal("\n") amount;
end
Key specifier - cust_id not specified in parameters.
Code in Transform:
type temporary_type=record
decimal(4) index;end; /*Temporary variable*/
temp :: initialize(in) =
begin
temp.index :: 0;
end;
temp :: scan(temp, in) =
begin
temp.index :: temp.index+1;
end;
out :: finalize(temp, in) =
begin
out.Sno :: temp.index;
out.* :: in.*;
end;
out :: key_change(in1, in2) =
begin
out :: in1.cust_id!=in2.cust_id;
end;
Output:
record
decimal("|") Sno;
decimal("|") cust_id;
string("|") cust_name;
decimal("|") bill_no;
decimal("\n") amount;
end
Example 3
To show input_select function
Input
record
decimal("|") cust_id;
string("|") cust_name;
decimal("|") bill_no;
decimal("\n") amount;
end
Code in Transform:
type temporary_type=record
decimal(4) index;end; /*Temporary variable*/
temp :: initialize(in) =
begin
temp.index :: 0;
end;
temp :: scan(temp, in) =
begin
temp.index :: temp.index+1;
end;
out :: finalize(temp, in) =
begin
out.Sno :: temp.index;
out.* :: in.*;
end;
out :: key_change(in1, in2) =
begin
out :: in1.cust_id!=in2.cust_id;
end;
out :: input_select(in) =
begin
out :: in.amount>1000;
end;
Output:
record
decimal("|") Sno;
decimal("|") cust_id;
string("|") cust_name;
decimal("|") bill_no;
decimal("\n") amount;
end
Example 3
To show output_select function
Input
record
decimal("|") cust_id;
string("|") cust_name;
decimal("|") bill_no;
decimal("\n") amount;
end
Code in Transform:
type temporary_type=record
decimal(4) index;end; /*Temporary variable*/
temp :: initialize(in) =
begin
temp.index :: 0;
end;
temp :: scan(temp, in) =
begin
temp.index :: temp.index+1;
end;
out :: finalize(temp, in) =
begin
out.Sno :: temp.index;
out.* :: in.*;
end;
out :: output_select(out) =
begin
out :: out.amount<1000;
end;
Output:
record
decimal("|") Sno;
decimal("|") cust_id;
string("|") cust_name;
decimal("|") bill_no;
decimal("\n") amount;
end
