4.8.7

FTP component

FTP Component

The FTP component is able to connect to FTP/SFTP servers and read or write files. It can act both as inbound component (read files from the server), and outbound component (write files to the server).

Basic configuration

The FTP component has the following basic configuration options:

Force inbound

Description

Settings this to Yes will force the component to turn from outbound to inbound and consume a file from the (S)FTP(S) server. This has the same behavior as putting the component below an Enrich component that is set to Override. This means that the current body will be overridden with the new body of the consumed file.

Remarks

  • Only available when the component is outbound.
  • Only one file will be consumed when the component is forced to be inbound.
  • This options only becomes visible when another component is connected on the left side of this component.

Protocol

Options

  • SFTP (default)
  • FTPS
  • FTP

Description

Specify whether the server is an FTP server, SFTP server or FTPS Server.

Host

Description

URL to the (S)FTP(S) server, e.g. ftp.example.org or IP address, e.g. 149.210.209.32.

Port

Description

Specify the (S)FTP(S) port if the server uses non-default ports. When left empty, defaults are FTP(S): 21, SFTP: 22.

Username

Description

Username used to log in on the (S)FTP(S) server.

Password

Description

Password used to log in on the (S)FTP(S) server.

Private Key file

Description

Location of a private key to log in on the SFTP server.

Remarks

  • Only available when the Protocol is set to SFTP.

Path

Description

Path to the target directory. In case of inbound this is the directory where Dovetail will be picking up files from, if used as outbound this is the directory where Dovetail will write files to.

Remarks

  • If you include a / at the end of the path, the component will try to copy the whole file path and not the file!

File Name

Description

Get specific file with this name when inbound or set file name when outbound.

Charset

Description

Specifies the encoding of the file.

Remarks

File Exist

Options

  • Override (default)
  • Append
  • Ignore
  • Fail

Description

Specify what to do if a file with the same name already exists.

Remarks

  • Only available when the component is outbound.

Delay

Description

Milliseconds to wait before the next poll of the file/directory.

Remarks

  • Only available when the component is inbound.

Advanced configuration

The FTP component has the following advanced configuration options:

Passive Mode

Options

  • Yes (default)
  • No

Description

Specify whether FTP should operate in active or passive mode.

Disconnect

Options

  • Yes (default)
  • No

Description

Whether or not to disconnect from the (S)FTP(S) server right after use. It will reconnect when the component is used again.

Remarks

  • Only available when the component is inbound.

Auto Create

Options

  • Yes (default)
  • No

Description

Automatically creates missing directories in the file path.

Remarks

  • Only available when the component is outbound.

Recursive

Options

  • Yes
  • No (default)

Description

When inbound, specifies whether Dovetail should process files only in the specified path (No) or in all subdirectories of the path too (Yes).

Remark

When set to Yes you must make sure that your exclude pattern will ignore the directory were you place the processed files, otherwise you will end up with a infinite loop.

Delete file

Options

  • Yes
  • No (default)

Description

If set to Yes, the file will be deleted after it is processed.

Remarks

  • Only available when the component is inbound.
  • When set to Yes, the Move option is ignored and hidden.

Move

Description

Relative path to folder where files should be moved to when done processing.

Create and move files into this folder when done processing. This should be a relative path. In order to move to higher paths use ../[folder name] or ../../[folder name].

Remarks

  • Only available when the component is inbound.
  • Only used and shown when Delete file is set to No.
  • Files can be moved to folders higher in the hierarchy with .., for example: ../[folder name] or ../../[folder name].
  • The default behavior is to create a directory named .dovetail and move everything into that directory when files are processed. However some (S)FTP(S) servers don't allow creating .\* files or directories. In this case, you need to change the name of the .dovetail directory to something that is accepted by the server.

Read lock

Options

  • none
  • changed

Description

If set to changed, only consumes the files if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written). It will wait until the file lock is granted.

Remarks

  • Only available when the component is inbound.

When set to changed:

  • It requires write permissions on the (S)FTP(S) server.
  • It uses file length or file modification timestamp to detect whether the file is currently being changed or not. It will at least use 1 second to check this and there will also be some time between each check, so consuming files will be slower.
  • It will require a lock on the latest file until a new file has been written to the (S)FTP(S) server. It will not be processed until a new file has been dropped at the ftp server.

Include

Description

Used to include files, if the filename matches the regular expressions pattern.

Remarks

  • Only available when the component is inbound.

Exclude

Description

Used to exclude files, if the filename matches the regular expressions pattern.

Remarks

  • Only available when the component is inbound.

Initial delay

Description

Milliseconds to wait before polling the file/directory starts.

Remarks

  • Only available when the component is inbound.

Producing large text files (> 10 MB)

When producing large text files like XML or JSON you might want to leave the charset option empty. This tells Dovetail that it should read the file as binary which means it does not have to read all the file's content into memory to validate the charset. This is better for it's performance and can prevent errors parsing the file later on in the flow.

Consuming folders with a lot of files

When the FTP component acts as a inbound component it can't pick up multiple files at the same times and process them. Every time the FTP Component pols the server it will create a list of all the available files and pick one to process. For this reason consuming a folder with a large amount of files can take more time than expected.

To improve the performance you could split up all the files over different subfolders, so you can poll each subfolder with a different FTP component.

Remarks

  • Authentication to SFTP servers can be provided either by specifying a password, or a path to a SSH private key file.
    • The private key file needs to be of the format OpenSSH.
  • You can use ${header.<headername>} and ${headers.<headername>} variables for all properties of the FTP component
← File componentGetVariable component →