FileWatcher Configuration | DataMystic
FileWatcher Configuration Reference
Configure FileWatcher to monitor folders for new or changed files and automatically trigger processing pipelines. Run as a desktop application or Windows service for unattended 24/7 operation.
Folder Monitoring
Define one or more watched folders. FileWatcher continuously monitors these folders and triggers actions when files match the configured conditions.
Watch Configuration Options
| Option | Type | Description |
|---|---|---|
WatchFolder |
String (path) | The folder path to monitor. Supports local paths, UNC network paths (\\server\share), and mapped drives. |
FileMask |
String | Wildcard pattern to match filenames. Multiple masks separated by semicolons. Examples: *.csv, *.txt;*.dat, report_*.xlsx |
ExcludeMask |
String | Wildcard pattern for files to exclude from monitoring. Example: *.tmp;~* |
IncludeSubfolders |
Boolean | When enabled, monitors all subfolders recursively. Default: False. |
MaxDepth |
Integer | Maximum subfolder depth when IncludeSubfolders is enabled. 0 = unlimited. Default: 0. |
PollInterval |
Integer (seconds) | How often to check the folder for changes. For network paths where filesystem notifications are unreliable, polling is used. Default: 5 seconds. |
UseNotifications |
Boolean | Use Windows filesystem change notifications (instant detection) instead of polling. Recommended for local folders. Default: True for local, False for network. |
Trigger Conditions
Define what file events trigger the action chain. Multiple conditions can be combined.
Event Types
| Trigger | Description |
|---|---|
FileCreated |
Triggers when a new file appears in the watched folder (including files copied or moved into it). |
FileModified |
Triggers when an existing file's contents are modified (last-write timestamp changes). |
FileRenamed |
Triggers when a file is renamed within the watched folder. |
FileDeleted |
Triggers when a file is removed from the watched folder. |
Condition Modifiers
| Option | Type | Description |
|---|---|---|
MinFileSize |
Integer (bytes) | Only trigger if file size is at least this many bytes. Useful to ignore empty placeholder files. Default: 0 (no minimum). |
MaxFileSize |
Integer (bytes) | Only trigger if file size is at most this many bytes. Default: 0 (no maximum). |
StableTime |
Integer (seconds) | Wait until the file size has not changed for this many seconds before triggering. Essential for large files that are still being written or uploaded. Default: 2 seconds. |
StableRetries |
Integer | Maximum number of stability checks. If the file is still changing after this many checks, skip it. Default: 10. |
DelayAfterTrigger |
Integer (seconds) | Wait this many seconds after the trigger condition is met before executing actions. Allows batching of multiple rapid changes. Default: 0. |
TimeWindow |
String (HH:MM-HH:MM) | Only trigger during the specified time window. Example: 08:00-18:00 (business hours only). Leave empty for 24/7 monitoring. |
DaysOfWeek |
String | Days of the week to monitor. Comma-separated: Mon,Tue,Wed,Thu,Fri. Leave empty for all days. |
Action Chains
When a trigger fires, FileWatcher executes a chain of actions in sequence.
If any action fails, subsequent actions can be skipped or the chain can continue
depending on the ContinueOnError setting.
Available Actions
| Action | Description | Key Parameters |
|---|---|---|
| Execute Program | Run an external program or script with the triggered file as a parameter. |
Program — Path to executableArguments — Command line arguments (use {file} placeholder for triggered file path)WorkingDir — Working directoryWaitForExit — Wait for program to finish before next actionTimeout — Maximum execution time in seconds
|
| Move File | Move the triggered file to a destination folder. |
Destination — Target folder pathOverwriteExisting — Overwrite if file existsCreateFolder — Create destination if it does not existPreserveTimestamp — Keep original file timestamps
|
| Copy File | Copy the triggered file to a destination folder. |
Destination — Target folder pathOverwriteExisting — Overwrite if file existsCreateFolder — Create destination if it does not exist
|
| Delete File | Delete the triggered file (permanently or to Recycle Bin). |
UseRecycleBin — Send to Recycle Bin instead of permanent deleteSecureDelete — Overwrite file contents before deletion
|
| Rename File | Rename the triggered file using a pattern. |
Pattern — New filename pattern (supports {date}, {time}, {counter}, {name}, {ext} placeholders)
|
| Email Notification | Send an email notification when a file triggers. |
SmtpServer — SMTP server addressSmtpPort — Port (default: 587 for TLS)From — Sender email addressTo — Recipient(s), semicolon-separatedSubject — Email subject (supports placeholders)Body — Email body (supports placeholders)AttachFile — Attach the triggered fileUseTLS — Enable TLS encryption
|
| Run TextPipe Filter | Process the triggered file through a TextPipe filter. |
FilterFile — Path to .fll filter fileOutputFolder — Where to write processed outputOverwriteOutput — Overwrite existing output files
|
| FTP Upload | Upload the triggered file to an FTP/SFTP server. See FTP Upload section for details. | See FTP Upload parameters below |
| Run Script | Execute a VBScript or PowerShell script inline. |
ScriptType — VBScript or PowerShellScriptBody — Inline script contentScriptFile — Path to external script file
|
Action Chain Options
| Option | Type | Description |
|---|---|---|
ContinueOnError |
Boolean | Continue executing subsequent actions even if one fails. Default: False (stop on first error). |
RetryCount |
Integer | Number of times to retry a failed action. Default: 0. |
RetryDelay |
Integer (seconds) | Seconds to wait between retry attempts. Default: 5. |
LogActions |
Boolean | Write detailed action execution log. Default: True. |
Filename Placeholders
The following placeholders are available in action parameters:
| Placeholder | Description | Example Output |
|---|---|---|
{file} |
Full path to the triggered file | C:\Data\report.csv |
{name} |
Filename without extension | report |
{ext} |
File extension (without dot) | csv |
{folder} |
Parent folder path | C:\Data |
{date} |
Current date (YYYY-MM-DD) | 2025-06-15 |
{time} |
Current time (HH-MM-SS) | 14-30-05 |
{counter} |
Auto-incrementing counter | 00001 |
{size} |
File size in bytes | 1048576 |
FTP Upload
Upload triggered files to remote FTP or SFTP servers. Supports passive and active mode, TLS encryption, and automatic directory creation.
FTP Connection Parameters
| Parameter | Type | Description |
|---|---|---|
Protocol |
String | Transfer protocol: FTP, FTPS (FTP over TLS), or SFTP (SSH File Transfer). Default: FTP |
Server |
String | FTP server hostname or IP address. |
Port |
Integer | Server port. Defaults: FTP=21, FTPS=990, SFTP=22. |
Username |
String | Login username. Use anonymous for anonymous FTP. |
Password |
String | Login password. Stored encrypted in the configuration file. |
RemotePath |
String | Remote directory path where files are uploaded. Supports placeholders. Example: /incoming/{date} |
PassiveMode |
Boolean | Use passive FTP mode (recommended for connections through firewalls). Default: True. |
ActiveMode |
Boolean | Use active FTP mode (server initiates data connection back to client). Default: False. |
CreateRemoteDir |
Boolean | Automatically create the remote directory if it does not exist. Default: True. |
TransferMode |
String | Transfer type: Binary or ASCII. Default: Binary |
TempExtension |
String | Upload with a temporary extension (e.g., .tmp) and rename after completion to prevent partial file processing on the server side. |
Timeout |
Integer (seconds) | Connection and transfer timeout. Default: 60. |
Retries |
Integer | Number of retry attempts on connection or transfer failure. Default: 3. |
RetryDelay |
Integer (seconds) | Seconds to wait between retries. Default: 10. |
Windows Service Mode
Run FileWatcher as a Windows service for unattended 24/7 operation. The service starts automatically with Windows and runs without requiring a user to be logged in.
Service Installation
Install the FileWatcher service from an elevated (Administrator) command prompt:
REM Install FileWatcher as a Windows service
"C:\Program Files\DataMystic\FileWatcher\FileWatcher.exe" /install
REM Uninstall the service
"C:\Program Files\DataMystic\FileWatcher\FileWatcher.exe" /uninstall
Service Configuration
| Option | Description |
|---|---|
Service Account |
By default, the service runs as Local System. For network folder access, configure a domain account with appropriate permissions via Windows Services (services.msc). |
Startup Type |
Set to Automatic (starts with Windows), Automatic (Delayed Start), Manual, or Disabled. Default after install: Automatic. |
Recovery Options |
Configure via services.msc Recovery tab: restart service on first failure, restart on second failure, take no action on subsequent failures. |
Configuration File |
The service uses the same configuration file as the desktop application: FileWatcher.ini in the application directory. Edit the file and restart the service to apply changes. |
Log Location |
Service logs are written to %ProgramData%\DataMystic\FileWatcher\Logs\ by default. |
Service Management Commands
REM Start the service
net start "DataMystic FileWatcher"
REM Stop the service
net stop "DataMystic FileWatcher"
REM Check service status
sc query "DataMystic FileWatcher"
REM Configure service to run under a domain account
sc config "DataMystic FileWatcher" obj= "DOMAIN\ServiceAccount" password= "password"
PowerShell Service Management
# Check service status
Get-Service "DataMystic FileWatcher"
# Restart service after configuration change
Restart-Service "DataMystic FileWatcher" -Force
# Set service to delayed automatic start
Set-Service "DataMystic FileWatcher" -StartupType AutomaticDelayedStart
# View service log
Get-Content "$env:ProgramData\DataMystic\FileWatcher\Logs\service.log" -Tail 50
Configuration Examples
Example 1: Process Incoming CSV Files with TextPipe
Monitor a folder for new CSV files, process them with TextPipe, then move to an archive folder:
// FileWatcher configuration - CSV processing pipeline
{
"watches": [
{
"name": "Incoming CSV Processor",
"watchFolder": "C:\\Data\\Incoming",
"fileMask": "*.csv",
"excludeMask": "*.tmp",
"includeSubfolders": false,
"triggers": {
"fileCreated": true,
"stableTime": 5,
"minFileSize": 100
},
"actions": [
{
"type": "RunTextPipeFilter",
"filterFile": "C:\\Filters\\csv_cleanup.fll",
"outputFolder": "C:\\Data\\Processed"
},
{
"type": "MoveFile",
"destination": "C:\\Data\\Archive\\{date}",
"createFolder": true
},
{
"type": "EmailNotification",
"to": "ops@company.com",
"subject": "CSV processed: {name}.{ext}",
"body": "File {file} was processed successfully at {time}."
}
],
"continueOnError": false,
"logActions": true
}
]
}
Example 2: FTP Upload with Stable File Detection
Wait for large files to finish writing, then upload to a remote server:
// FileWatcher configuration - FTP upload pipeline
{
"watches": [
{
"name": "Report FTP Upload",
"watchFolder": "D:\\Reports\\Outgoing",
"fileMask": "*.pdf;*.xlsx",
"includeSubfolders": false,
"triggers": {
"fileCreated": true,
"stableTime": 10,
"stableRetries": 20,
"timeWindow": "18:00-06:00",
"daysOfWeek": "Mon,Tue,Wed,Thu,Fri"
},
"actions": [
{
"type": "FTPUpload",
"protocol": "FTPS",
"server": "ftp.partner.com",
"port": 990,
"username": "upload_user",
"password": "(encrypted)",
"remotePath": "/incoming/{date}",
"passiveMode": true,
"createRemoteDir": true,
"tempExtension": ".tmp",
"transferMode": "Binary",
"retries": 3,
"retryDelay": 30
},
{
"type": "MoveFile",
"destination": "D:\\Reports\\Sent\\{date}",
"createFolder": true
}
]
}
]
}
Example 3: Multi-Step Document Processing Pipeline
Complex pipeline: detect new Word documents, process with WordPipe, convert with TextPipe, and notify:
// FileWatcher configuration - Multi-step document pipeline
{
"watches": [
{
"name": "Document Processing Pipeline",
"watchFolder": "\\\\fileserver\\incoming\\documents",
"fileMask": "*.docx;*.doc",
"includeSubfolders": true,
"maxDepth": 2,
"useNotifications": false,
"pollInterval": 10,
"triggers": {
"fileCreated": true,
"fileModified": true,
"stableTime": 3,
"minFileSize": 1024
},
"actions": [
{
"type": "ExecuteProgram",
"program": "C:\\Program Files\\DataMystic\\WordPipe\\WordPipe.exe",
"arguments": "/input:\"{file}\" /findlist:\"C:\\Config\\updates.txt\" /inplace /silent",
"waitForExit": true,
"timeout": 120
},
{
"type": "CopyFile",
"destination": "C:\\Processed\\{date}",
"createFolder": true
},
{
"type": "EmailNotification",
"to": "admin@company.com;compliance@company.com",
"subject": "[FileWatcher] Document processed: {name}.{ext}",
"body": "Document {file} ({size} bytes) was processed at {date} {time}.",
"attachFile": false
}
],
"continueOnError": true,
"retryCount": 2,
"retryDelay": 10,
"logActions": true
}
]
}
Example 4: Service Mode Setup Script
Complete PowerShell script to install, configure, and verify the FileWatcher service:
# FileWatcher Service Installation and Configuration
# Run as Administrator
$installPath = "C:\Program Files\DataMystic\FileWatcher"
$serviceName = "DataMystic FileWatcher"
# Step 1: Install the service
Write-Host "Installing FileWatcher service..."
Start-Process -FilePath "$installPath\FileWatcher.exe" -ArgumentList "/install" -Wait -Verb RunAs
# Step 2: Configure service account (for network folder access)
$credential = Get-Credential -Message "Enter service account credentials"
sc.exe config $serviceName obj= $credential.UserName password= $credential.GetNetworkCredential().Password
# Step 3: Set startup type to Automatic (Delayed Start)
Set-Service $serviceName -StartupType AutomaticDelayedStart
# Step 4: Configure recovery options (restart on failure)
sc.exe failure $serviceName reset= 86400 actions= restart/5000/restart/10000/restart/30000
# Step 5: Start the service
Start-Service $serviceName
# Step 6: Verify service is running
$svc = Get-Service $serviceName
if ($svc.Status -eq 'Running') {
Write-Host "FileWatcher service is running successfully." -ForegroundColor Green
} else {
Write-Host "ERROR: Service failed to start. Status: $($svc.Status)" -ForegroundColor Red
Get-EventLog -LogName Application -Source $serviceName -Newest 5
}