Mapping Activities - CaO_Data

Activities

Use the Activities on the right console of the Edit Mapping page to perform a number of activities on your mapping, including:

Run Map executes the map with any changes that you have made and saved to your mapping.
Delete Map removes the map from your instance.

You will be asked if you are sure that you want to delete this mapping before it is removed from the system, as shown in the warning image below.

Import Changes updates the mapping to include changes in the source data elements.

As this may overwrite custom changes that you have made to the mapping, you will be asked if you are sure that you want to import changes before continuing, as shown in the warning image below.

Source SQL opens the SQL query editor in the center console, described in detail below. Only available for database sources.
File Preview (file sources only) provides a window for viewing a sample set of data for delimited files.
Source Filters (Parquet sources only) opens an editor where you enter a filter condition, written like a SQL WHERE clause, to limit the rows read from the Parquet files. The saved filter is applied every time the mapping runs. See Source Filters below.
Target Properties allows you to add more advanced functionality to your mapping, including adding pre and/or post processing scripts that run existing SQL functions and adding advanced target table properties. See Target Properties below for more details.
Job History shows the Mapping History of past executions of this mapping in the center console. Click View Map in the Activities panel to return to the mapping.

Source SQL

Users can review the source SQL query that was generated by clicking the Source SQL button. The SQL query editor shows you the query and lets you execute it with the Run button, showing the results beneath the SQL statement. Click Save in the Activities panel to keep changes to the query, or Cancel to discard them.

Source SQL History

Every change to a mapping’s custom source SQL is recorded. Click the History button in the Source SQL panel header to see each version of the query, who changed it, and when.

The history list shows the most recent change first. Each row has:

Column	Description
Update Date	When the change was saved (UTC).
Update User	The user who saved the change.
Change Type	How the version came to be — see below.
Note	Context recorded by Kodda for system-generated entries.
Preview	The first line or so of the query.

Change types you may see:

create — the first custom SQL saved for the mapping.
update — a later edit saved from the Source SQL editor.
base_generated — the SQL Kodda generated automatically before the first custom override. Recorded so you can always get back to the original.
base_rebuild — custom SQL was cleared because the mapping was regenerated.
delete — the custom SQL was removed.

View a version

Click View on a row to open that version in full, with its date, user, and change type.

Compare two versions

Tick the checkbox on any two rows and click Compare (the button appears once two rows are selected). The two versions open side by side, oldest on the left, with differences highlighted.

History is read-only. To return to an earlier version, open it with View, copy the SQL, paste it into the Source SQL editor, and Save. That save is recorded as a new update entry, so the full trail is preserved.

Source SQL history is also available through the REST API at GET /api/v1/map/<map_id>/cust_sql_history, which returns the same entries as JSON.

Source Filters

For Parquet sources, click Source Filters in the Activities panel to enter a filter condition. Kodda applies the condition while it reads the Parquet files, so only the matching rows are extracted. If the mapping is incremental, the incremental filter is applied as well.

The condition is written like a SQL WHERE clause; the word WHERE at the start is optional. For example:

region = 'EMEA' and order_date >= '2024-01-01' and status not in ('cancelled', 'test')

Supported syntax:

Syntax	Example
Comparisons: `=`, `!=` or `<>`, `>`, `<`, `>=`, `<=`	`quantity > 0`
`AND`, `OR`, `NOT` and parentheses	`(a = 1 or b = 2) and not c = 3`
`IN` and `NOT IN`	`country in ('US', 'CA')`
`BETWEEN` and `NOT BETWEEN`	`amount between 10 and 100`

The left side of each comparison must be a plain column name (letters, numbers and underscores). Functions, calculations and quoted column names are not supported.
Put text values in quotes. A quoted value in the form 'YYYY-MM-DD' is treated as a date, and 'YYYY-MM-DD HH:MM:SS' as a timestamp.
Unquoted numbers, true, false and null are treated as those values.

Target Properties

Click Target Properties in the Activities panel to open the target settings in the center console. There are two tabs. Click Save in the Activities panel to keep your changes or Cancel to discard them.

Target Table Properties

Properties are specific settings available for your target system, grouped by Property Type (for example DDL or Load). Choose a Property Type, then a Property Name, and click + to add it; it then appears in the list below where you can set its value. Each property that is available for a target system is listed in the tables at the end of this page.

Target Table Scripts

Pre-Delivery Scripts is used to enter SQL statements or scripts that will run on your target before data is delivered to the target table. Post-Delivery Scripts run after delivery. These must use syntax native to your target.

Advanced Target Table Properties: Delimited File

Property Type	Property Label	Default	Valid Values	Description
Target	Create Subfolder	TRUE		Creates a subfolder matching the target table name and writes the file there.
Target	Create Empty File	FALSE		Create file if there are no records to write to the file.
Target	File Extension	txt		Overrides the target file extension specified in the connection.
Target	Dated Filename	Date	Date,Date/Time,None	Appends the date(time) to the end of the file name before the file part and extension
Target	Rows Per File	10000		Sets the maximum number of rows per file. When set the file part will be placed before the extension in the name.
File	Contains Header	TRUE		Used to override the file connection property and include/exclude a file header.
File	Row Separator	LF	CR/LF,LF,CR	Used to set the target file row separator in the target output file(s).
File	Delimiter	,		Used to set the target file column separator in the target output file(s).
File	File Encoding	utf-8	utf-8,utf-16,ascii	Specifies the default encoding to use when writing the arget output file(s).
Target	Sort By Columns		<source columns>	Data will be sorted by the columns selected.

Advanced Target Table Properties: MySQL

Property Type	Property Label	Default	Valid Values	Description
DDL	Preserve Nullability	FALSE		Matches the nullability of the source column when creating the target table.
Load	Microbatch Load	FALSE		Loads the target table in microbatches using the same batch size defined for the source extract.
Load	Null as Empty	FALSE		Replaces null values with empty strings when inserting to a non-nullable column
Load	Hard Delete	FALSE		When change tracking is used, removes records from the target that have been deleted in the source.
Load	Push Down	FALSE		Pushes the data processing down to the target database. This only works when the source and target database are the same.
Load	Skip Work Table	FALSE		When enabled and the load type is Full Refresh, data is loaded directly into the target table, bypassing the work table.
DDL	PK Columns		<target columns>	The columns used to create the primary key on the target.

Advanced Target Table Properties: Oracle

Property Type	Property Label	Default	Valid Values	Description
DDL	Preserve Nullability	FALSE		Matches the nullability of the source column when creating the target table.
Load	Microbatch Load	FALSE		Loads the target table in microbatches using the same batch size defined for the source extract.
Load	Hard Delete	FALSE		When change tracking is used, removes records from the target that have been deleted in the source.
Load	Push Down	FALSE		Pushes the data processing down to the target database. This only works when the source and target database are the same.
Load	Load Method	Direct Path	Direct Path,Execute Many	Determines the method used to load data to the Oracle work table. Is not used when pushdown optimization is enabled.
Load	Skip Work Table	FALSE		When enabled and the load type is Full Refresh, data is loaded directly into the target table, bypassing the work table.
DDL	PK Columns		<target columns>	The columns used to create the primary key on the target.

Advanced Target Table Properties: PostgreSQL

Property Type	Property Label	Default	Valid Values	Description
DDL	Preserve Nullability	FALSE		Matches the nullability of the source column when creating the target table.
Load	Microbatch Load	FALSE		Loads the target table in microbatches using the same batch size defined for the source extract.
Load	Null as Empty	FALSE		Replaces null values with empty strings when inserting to a non-nullable column
Load	Hard Delete	FALSE		When change tracking is used, removes records from the target that have been deleted in the source.
Load	Bulk Load	FALSE		Used to enable fast bulk loading (using COPY FROM) to the target.
Load	Push Down	FALSE		Pushes the data processing down to the target database. This only works when the source and target database are the same.
Load	Skip Work Table	FALSE		When enabled and the load type is Full Refresh, data is loaded directly into the target table, bypassing the work table.
DDL	PK Columns		<target columns>	The columns used to create the primary key on the target.

Advanced Target Table Properties: Redshift

Property Type	Property Label	Default	Valid Values	Description
DDL	Preserve Nullability	FALSE		Matches the nullability of the source column when creating the target table.
Load	Null as Empty	FALSE		Replaces null values with empty strings when inserting to a non-nullable column
Load	Accept Invalid Chars	FALSE		When enabled and doing a bulk insert replaces invalid utf-8 characters with a ^. Only applies to varchar columns
Load	Hard Delete	FALSE		When change tracking is used, removes records from the target that have been deleted in the source.
Load	Bulk Load	FALSE		Used to enable fast bulk loading to the target. Requires an S3 bucket to be setup on the target connection.
Load	Rows Per File	0		Specifies the number of rows that will be written to each file. A value of 0 indicates there is no limit.
DDL	Distribution Style	EVEN	EVEN,KEY,ALL	Defines the data distribution style for the table. Even-Distributes the data evenly across all nodes. All - creates a copy of the table on all nodes. Key - distributes data across the defined key.
DDL	Sort Style	COMPOUND	COMPOUND,INTERLEAVED	Adds a sort key to the target table. Adding this property also shows a Sort Columns list; choose the target columns for the sort key, in order. No sort key is created unless at least one column is selected. COMPOUND sorts by the columns in the order listed and suits queries that filter on the first columns; INTERLEAVED gives each column equal weight. Redshift allows up to 400 COMPOUND or 8 INTERLEAVED sort key columns. The sort key is set only when Kodda creates the table, so use Recreate Target to apply a change to an existing table.
Load	Compress Files	TRUE		Used to determine whether to compress files that are loaded into the Redshift table. When Bulk Load is set to TRUE(enabled) this defaults to TRUE(enabled).
Load	Compression Level	6		Specifies the amount of compression on the file. 0 is the fastest with no compression 9 is the slowest with the most compression. When Compress Files is enabled this is set to 6 by default.
Load	Push Down	FALSE		Pushes the data processing down to the target database. This only works when the source and target database are the same and a hash target data type is not selected.
DDL	PK Columns		<target columns>	The columns used to create the primary key on the target.

Advanced Target Table Properties: Snowflake

Property Type	Property Label	Default	Valid Values	Description
DDL	Preserve Nullability	FALSE		Matches the nullability of the source column when creating the target table.
Load	Null as Empty	FALSE		Replaces null values with empty strings when inserting to a non-nullable column
Load	Hard Delete	FALSE		When change tracking is used, removes records from the target that have been deleted in the source.
Load	Rows Per File	0		Specifies the number of rows that will be written to each file. A value of 0 indicates there is no limit.
DDL	Cluster Columns		<target columns>	The columns used to sort the data in the Snowflake micro-partition..
Load	Push Down	FALSE		Pushes the data processing down to the target database. This only works when the source and target database are the same.
DDL	PK Columns		<target columns>	The columns used to create the primary key on the target.

Advanced Target Table Properties: SQL Server

Property Type	Property Label	Default	Valid Values	Description
Load	Bulk Load	FALSE		Used to enable fast bulk loading (using BCP) to the target.
DDL	Create Clustered PK	TRUE		Used to determine the structure of the primary key index on the target table when the table is first created.
DDL	Create PK After Load	FALSE		When enabled and a target table is created as part of the execution. The primary key will be created after the data has been loaded to the table. This may help to speed up the initial load of a table.
DDL	Preserve Nullability	FALSE		Matches the nullability of the source column when creating the target table.
Load	Microbatch Load	FALSE		Loads the target table in microbatches using the same batch size defined for the source extract.
Load	Null as Empty	FALSE		Replaces null values with empty strings when inserting to a non-nullable column.
Load	Hard Delete	FALSE		When change tracking is used, removes records from the target that have been deleted in the source.
Load	Push Down	FALSE		Pushes the data processing down to the target database. This only works when the source and target database are the same.
Load	Skip Work Table	FALSE		When enabled and the load type is Full Refresh, data is loaded directly into the target table, bypassing the work table.
DDL	PK Columns		<target columns>	The columns used to create the primary key on the target.
DDL	Table Compression		PAGE,ROW	Type of Compression to use for target table.

Advanced Target Table Properties: Informix

Property Type	Property Label	Default	Valid Values	Description
Load	Hard Delete	FALSE		When change tracking is used, removes records from the target that have been deleted in the source.
Load	Push Down	FALSE		Pushes the data processing down to the target database. This only works when the source and target database are the same.
Load	Skip Work Table	FALSE		When enabled and the load type is Full Refresh, data is loaded directly into the target table, bypassing the work table.
DDL	PK Columns		<target columns>	The columns used to create the primary key on the target.
Load	Microbatch Load	FALSE		Loads the target table in microbatches using the same batch size defined for the source extract.

​Activities

​Source SQL

​Source SQL History

​View a version

​Compare two versions

​Source Filters

​Target Properties

​Target Table Properties

​Target Table Scripts

​Advanced Target Table Properties: Delimited File

​Advanced Target Table Properties: MySQL

​Advanced Target Table Properties: Oracle

​Advanced Target Table Properties: PostgreSQL

​Advanced Target Table Properties: Redshift

​Advanced Target Table Properties: Snowflake

​Advanced Target Table Properties: SQL Server

​Advanced Target Table Properties: Informix

Activities

Source SQL

Source SQL History

View a version

Compare two versions

Source Filters

Target Properties

Target Table Properties

Target Table Scripts

Advanced Target Table Properties: Delimited File

Advanced Target Table Properties: MySQL

Advanced Target Table Properties: Oracle

Advanced Target Table Properties: PostgreSQL

Advanced Target Table Properties: Redshift

Advanced Target Table Properties: Snowflake

Advanced Target Table Properties: SQL Server

Advanced Target Table Properties: Informix