Scheduled data imports

• Database connection / SQL
• CSV file import
• Parsing XML data files
• Converting emails to data

Database connection / SQL

Databases fields are mapped to solution fields by their SQL names.

The SQL import allows for incremential imports, by adding a datetime filter on the records extracted.

Connections for the remote database is set up in: Designer > Integration > DB Connections

CSV file import

CSV imports are handled much in the same way as SQL connections using af CSV database driver.

The differences are

• Use the filename as substitute for the table name
• Files have to be placed in the folder set by the configuration: folderIntegrationCSV
• Support for very simple WHERE clauses
• No support for aggregate functions

Parsing XML data files

XML fields are mapped to solution fields by xpath expressions.

XML integration also handles task splitting and handling. Each file is split into multiple smaller requests, that are logged and handled individually. In that way an error in a single line, will not prevent the complete batch from executing. The task split is done by an xpath expression.

Files are split in the following folder structure

• Tasks that are awaiting processing
• Tasks that have been processed
  • Tasks processed with succes
  • Tasks processed with errors

This allows for easy resubmission of problem data ("Tasks processed with errors"), by copying the file back to the preocessing folder ("Tasks awaiting processing").

Files have to be placed in the folder set by the configuration: folderIntegrationXML

Converting emails to data

The email import converts new emails sent to a certain account.

A few special email "values" can be mapped to solution fields

• Email: Senders address
• Subject: Email subject title line (text)
• Content: Text payload in email (rich text)
• Attatchments: Files attached to the email

After processing the emails they are marked as read at the server - regardless of succes/error status. Reprocessing can be invoked by changing the read status of the email in question.