Design Workflow
Last updated
Last updated
Use the Design Workflow screen to control the flow of mappings, custom scripts and other workflows.
Access the Design Workflow screen on the left navigation menu or in the drop down menu via the Design tab in the main header.
To find objects to add into your workflow, use the Available Components section to either Search for an object or traverse via the folder structure.
The following object types can be added to a workflow:
Utility provides two additional options to use in making advanced data pipelines, including:
NOTE: script syntax must be native to the Script Connection selected in your scripts Component Properties
Email: This utility will allow you to send email messages within the workflow. Additional, you may attach a dataset which will be included as a CSV file in the email message. If the attached dataset contains more than 5,000 records only the first 5,000 records will be included in the attached file.
All workflows begin with a Start icon followed by any of the Available Components described above which are dragged onto the palette.
This action draws the flow line between the two objects as shown below by the blue lines below.
Circular references are not allowed at this time. If a circular reference exists you must correct it prior to saving or running the workflow.
You perform the following actions by clicking on corresponding buttons, located in the far right panel of the screen:
Run Workflow to execute the workflow.
Save Workflow to save all changes to the workflow.
Delete Workflow to delete a saved the workflow.
Export Workflow to export the workflow and all of dependencies. This is useful when operating in separate development and production instances.
Job History to show all current and prior runs of a saved workflow.
Expand the collapsible area used to enter the following details, shown above.
Workflow Name to view or edit the name.
Continue on Error (toggle) can be used to create a continuous workflow if any component should fail.
Expand the collapsible area to change the messaging defaults as follows:
On Error (toggle) - if set, an email is sent if an error occurs in the workflow.
On Success (toggle) - if set, an email is sent if the workflow completes successfully.
Message To - enter the email address to receive the error or success message. Use comma separated emails to enter multiple addresses.
Message From - enter the email address from which to send error or success messages. If left blank, this will use the default email in your configuration. See setup.cfg for details.
Include Log (toggle) - if set, any messages will attach the log file directly, so the recipient need not log into the application and search for the log file.
Click on a component of your workflow to display appropriate Component Properties for that type. Read below to learn how to view and edit all of your workflow components properties.
Map and Workflow components provide you the options, fields and buttons shown in the image above and described below.
Enabled (toggle) allows you to set a component to be skipped by turning off this option, which defaults to on (as shown).
Title allows you edit the name of this component in your workflow. Names will become important when you have larger flows.
Type is a display only field that shows you the workflow component type, either map or workflow.
Mapping Name is a display only field that shows you the name of the mapping or workflow.
Edit Map enables you to link directly to the component Map or Working details. Click to jump to the Edit Mapping or Design Workflow screen for that component.
Remove allows you to remove the map or workflow component from this workflow. The actual component is not deleted from the system, it is only removed from this workflow.
In the left panel, you can start the process of creating a new workflow by clicking the New Workflow button, or you can browse existing workflows to view or edit. If browsing, you can use the Search field to perform a web-like search of your existing workflows, or you can use the folder structure to traverse your Available Workflows.
Mapping allows you to add any existing mapping to your workflow. As shown in the image below, when traversing folder structures you will see mappings broken out by their type, connection and schema. When placed on workflows, mappings show as the icon.
Workflow allows you to add any existing workflow to your workflow. You can use this nested capability of workflows to build advanced data pipelines. When placed on workflows, workflows show as the icon.
SQL Script which lets you embed custom SQL statements directly into your workflow, which show as the icon.
Anchor which tells your workflow to wait for all incoming components to complete before proceeding, which show as the icon.
Components are linked together in a flow by clicking on the gray arrow shown to the right of one component and then moving your mouse to that sameicon shown to the left of another and clicking again.
