Design Workflow

Use the Design Workflow screen to control the flow of mappings, SQL scripts, email notifications and other workflows.

​

Navigation

Access the Design Workflow screen from the left navigation menu or from the Design drop-down in the main header. The screen has three areas:

• Left panel — the Library of existing workflows and the Components you can add to a workflow.
• Center panel — the design palette where the workflow is drawn.
• Right panel — Activities, Workflow Properties, Workflow Messaging and Component Properties.

​

Create a New Workflow

​

Library

In the left panel, click + New Workflow to start a new workflow, or browse the Library tab to open an existing workflow to view or edit. Use the Search field to search your existing workflows by name, or expand the folder structure to find them. Double-click a workflow to open it on the palette.

​

Components

The Components tab lists the objects that can be added to a workflow. Use the Search components field to find an object by name, or expand the tree. Drag a component from the tree onto the palette to add it to the workflow. The following object types can be added to a workflow:

• Mapping — any existing mapping. Mappings are grouped in the tree by target connection, then schema. When placed on the palette a mapping shows as the icon.
• Workflow — any existing workflow. Nesting workflows lets you build advanced, multi-level data pipelines. A nested workflow shows as the icon.
• Utility — additional components for building advanced pipelines:
  • SQL Script embeds a custom SQL statement directly in the workflow and shows as the icon. The script syntax must be native to the Connection selected for the script.
  • Anchor waits for all incoming components to complete before proceeding and shows as the icon. Use an anchor to fan a set of parallel mappings back into a single path.
  • Email sends an email message from within the workflow and shows as the icon. Optionally, a dataset returned by a SQL query can be attached as a CSV or XLSX file. If the attached dataset contains more than 5,000 records only the first 5,000 are included.

​

Workflow Design Palette

Every workflow begins with a Start component. Drag Components from the left panel onto the palette and link them together to define the order of execution.

• To link two components, click the grey output arrow on the right of one component, move the mouse to the input arrow on the left of another component and click again.
• A flow line is drawn between the two components. Components with multiple incoming lines wait for all of their predecessors to finish before running; components with multiple outgoing lines start their successors in parallel.
• Click a component to select it and show its Component Properties. Drag a component to reposition it.

Use the controls in the lower right corner of the palette to navigate large workflows:

• Search operators — type part of a component name to locate it on the palette.
• + / − — zoom in and out. You can also scroll with the mouse wheel and drag the background to pan.
• The reset button returns the palette to the default zoom, centered on the workflow.

​

Activities

The buttons in the far right panel perform the following actions:

• Run Workflow executes the workflow. The workflow must be saved first.
• Save Workflow saves all changes to the workflow.
• Delete Workflow deletes the saved workflow.
• Export Workflow exports the workflow and all of its dependencies (mappings and nested workflows) as a .kexp file. Connections are not exported; they are matched to existing connections when the file is imported. This is useful when promoting between separate development and production instances. See Migrate.
• Job History shows all current and prior runs of the workflow. See Workflow History.

​

Workflow Properties

Expand the collapsible area to view or edit the following details:

• Workflow Name — view or edit the name of the workflow.
• Continue On Error (toggle) — when on, the workflow continues to run the remaining components if a component fails. When off (the default) the workflow stops at the first failure.
• Reset Incrementals (toggle) — when on, the Next Incremental Value of every incremental mapping in the workflow is cleared before the mapping runs, forcing a full reload of the source. After the workflow completes successfully the toggle is switched off automatically so the next run is incremental again.
• Created On / Created By / Updated On / Updated By — audit information for the workflow.
• Workflows — links to any parent workflows that include this workflow as a component.

​

Workflow Messaging

Expand the collapsible area to change the notification settings for the workflow:

• On Error (toggle) — if set, an email is sent when the workflow fails.
• On Success (toggle) — if set, an email is sent when the workflow completes successfully.
• To — the email address that receives the error or success message. Separate multiple addresses with commas.
• From — the email address the message is sent from. If left blank the default address from your configuration is used.
• Include Log (toggle) — if set, the workflow log file is attached to the message so the recipient does not need to log in to the application to find it.

​

Component Properties

Click a component on the palette to display its Component Properties. The properties shown depend on the component type.

​

Mapping components

• Enabled (toggle) — turn off to skip this component when the workflow runs. Defaults to on.
• No Increment On Failure (toggle) — for incremental mappings. When on, if the workflow ends in a failed status the mapping’s Next Incremental Value is restored to the value it had before the workflow ran, so the next run picks up the same records again.
• Title — the label shown on the palette. Defaults to the mapping name; edit it to make large workflows easier to read.
• Type — display only; map.
• Mapping Name — display only; the name of the mapping.
• Edit Map — opens the mapping in the Edit Mapping screen.
• Remove — removes the component from this workflow. The mapping itself is not deleted.

​

Workflow components

• Enabled, Title and Remove work as described for mapping components.
• Type — display only; wf.
• Workflow Name — display only; the name of the nested workflow.
• Edit Workflow — opens the nested workflow in the Design Workflow screen.

​

SQL Script components

• Enabled, Title and Remove work as described for mapping components.
• Type — display only; script.
• Edit Script — opens the script editor in place of the palette. Select the Connection the script will run against, enter the SQL in the editor and click Save to return to the palette, or Cancel to discard your changes.

​

Email components

• Enabled, Title and Remove work as described for mapping components.
• Type — display only; email.
• Compose Message — opens the message editor in place of the palette.

Message Contents

• From — the sending address. If left blank the default address from your configuration is used.
• To — one or more recipient addresses, separated by commas.
• Subject and Message — the subject line and body of the email.

Attach Dataset (optional)

• Connection — the connection the SQL query runs against.
• Dataset Name — the file name of the attachment (without extension).
• Send If Empty (toggle) — defaults to on. When off, the email is not sent if the query returns no records.
• File Format — csv or xlsx.
• SQL Query — the query whose result set is attached. Only the first 5,000 records are included.

Click Save to return to the palette, or Cancel to discard your changes.