Quick Start¶

Run the install script¶

$ curl https://cloud.swiftstack.com/v1/AUTH_gowf/gowf/install-gowf.sh | bash
Note

The install script currently supports

Mac OS

Linux OS

FreeBSD jail on FreeNas

For Windows users, please download the binary file in below,

Windows - gowf.exe

If you want to run install script on a FreeBSD jail on FreeNas, here is a reference command that runs on FreeNAS 11.2-RELEASE.

root@freenas[~]# iocage create --release 11.2-RELEASE --name gowf.jail \
                        boot="on" \
                        allow_raw_sockets="1" \
                        ip4_addr="em0|10.0.2.18/24" \
                        resolver="nameserver 168.95.1.1;nameserver 168.95.192.1" \
                        defaultrouter="10.0.2.2"

root@gowf:~ # pkg update && pkg install curl sudo bash

root@gowf:~ # curl https://cloud.swiftstack.com/v1/AUTH_gowf/gowf/install-gowf.sh | bash

Generate and modify the configuration file¶

Run gowf config to generate a configuration file and also it shows you the locations of binary, configuration and log files.

$ gowf config
# Config: /etc/gowf/gowf.conf
# Binary: /bin/gowf
# Log: /var/log/gowf/gowf.log

Generate config file: /etc/gowf/gowf.conf
Swift endpoint: https://swift.example.com/auth/v1.0
User name: charles
Password:
  -> Connection verification: Passed

Monitor folder: /tmp/source
Remote container: mylocalfolder
  -> Updated config: /etc/gowf/gowf.conf
Note

You might see different locations when you run GoWF on different operating systems.

MacOS:

Config: /Users/<username>/gowf/gowf.conf

Log: /Users/<username>/gowf/log/gowf.log

Linux and FreeNAS:

Config: /etc/gowf/gowf.conf

Log: /var/log/gowf/gowf.log

Windows:

Config: C:/<the_folder_gowf_binary_located>/config/gowf.conf

Log: C:/<the_folder_gowf_binary_located>/log/gowf.log

And then you can check the configuration file to see if that matches your expectation.

$ cat /etc/gowf/gowf.conf
[global]
user = charles
auth = https://swift.example.com/auth/v1.0
key =

concurrency = 2

# default segment_size is 100MB
# Allow suffix size string (B, KB, MB, GB)
segment_size = 100MB
segment_container_prefix = .segment_

recursive = False
preserve_path = True

[/tmp/source]
storage_policy = Standard-Replica
container = mylocalfolder

# `archive` mode:
#       Once the local files uploaded to swift, it will
#       try to delete these local files in a certain time that
#       you set in `keep_local_files`
# `sync` mode:
#       Sync local files to Swift, and won't delete local files.
#
mode = sync

# How long you would like to keep these files in local
# This option is only valid when you set `mode = archive`
# y: year, w: week, d: day, h: hour, m: minute, s: second
# keep_local_files = 30m

# split with comma
# file_patterns = *.txt, *.log
# file_patterns = *abc*

# expired remote object in `expired_after`
# y: year, w: week, d: day, h: hour, m: minute, s: second
# expired_after = 60d

# Metadata for objects
# metadata = key1:val1, key2:val2

$ ls -al /etc/gowf/.vaults/
total 8
drwx------   3 charles  staff    96 Apr 26 15:58 ./
drwxr-xr-x  38 charles  staff  1216 May  6 15:53 ../
-rw-------   1 charles  staff   537 Apr 26 15:58 ac8c8b5883edbb66f8ea6c76f7dab3bfba48aabe

Note

In the example above, the value of key is empty due to GoWF stored the key to an encrypted file under .vaults directory in the same folder that gowf.conf stored.

If you already have the key in gowf.conf and want to use an encrypted file to store the key, please check here to get more details.

Install and start the GoWF service¶

Currently we support these service scripts in below,

launchctl on MacOS.

### Install/uninstall system script
$ sudo gowf service install
$ sudo gowf service uninstall

### Start/stop system service
$ sudo gowf service start
$ sudo gowf service stop

### Check service
$ launchctl list |grep gowf

Systemd and SysVinit on Linux distribution.

### Install/uninstall system script for Linux
$ sudo gowf service install
$ sudo gowf service uninstall

### Start/stop system service
$ sudo gowf service start
$ sudo gowf service stop

### for systemd
$ sudo systemctl start gowf
$ sudo systemctl stop gowf
$ sudo systemctl status gowf

### for sysvinit on Linux and rc script on FreeBSD
$ sudo service gowf start
$ sudo service gowf stop
$ sudo service gowf status

Windows Service on Windows.

### Install/uninstall Windows service with Administrator
$ gowf.exe service install
$ gowf.exe service uninstall

### Start/stop Windows service
$ gowf.exe service start
$ gowf.exe service stop

You can find the service name called Go Watch Folder for swift in Service management window.
Here is the detail of Go Watch Folder for swift service.

rc script for FreeBSD jail.

### Install script will create a rc script for FreeBSD jail.

### rc script on FreeBSD jail
root@gowf:~ # sudo service gowf start
root@gowf:~ # sudo service gowf stop
root@gowf:~ # sudo service gowf status

### Start/stop jail
root@freenas[~]# iocage start <gowf-jail-name>
root@freenas[~]# iocage stop <gowf-jail-name>

### Add the shared folder that you want to monitor in GoWF
root@freenas[~]# iocage fstab -a <gowf-jail-name> "/mnt/test /mnt/test nullfs rw 0 0"
Successfully added mount to gowf's fstab

root@freenas[~]# iocage fstab -l <gowf-jail-name>
+-------+--------------------------------------------------------------+
| INDEX |                         FSTAB ENTRY                          |
+=======+==============================================================+
| 0     | /mnt/test /mnt/iocage/jails/gowf/root/mnt/test nullfs rw 0 0 |
+-------+--------------------------------------------------------------+

### Start FreeBSD jail and upload files from /mnt/test to remote Swift cluster
root@freenas[~]# iocage start <gowf-jail-name>

Quick check the GoWF log¶

$ tail -f /var/log/gowf/gowf.log

Help message¶

$ gowf -h
Go WatchFolder

Usage:
    gowf --config CONFIG_FILE [--config CONFIG_FILE]...
         [--log-file LOG_FILE]
         [--log-level LOG_LEVEL]
         [--log-rotate]
         [--stats-folder FOLDER]
         [--exporter-port EXP_PORT]
         [--dedup|--purgededup|--statdedup]
         [--download CONTAINER OBJECT PATH]
         [--list CONTAINER]
         [-o | --once]
    gowf -u | --update
    gowf -h | --help
    gowf --version
    gowf service install | uninstall | start | stop
    gowf config

Options:
    --config CONFIG_FILE                                            A configuration file.
                                                                    To run multiple config files, please try "--config <config1> --config <config2>".
    --log-file LOG_FILE                                             Log file  [default: gowf.log].
    --log-level LOG_LEVEL                                           Log level (DEBUG|INFO) [default: INFO].
    --log-rotate                                                    Rotate log when file size reaches 512MB and the max backup is 20.
    --stats-folder FOLDER                                           Store statistic output in the folder [default: ].
    --exporter-port EXP_PORT                                        Enable Prometheus exporter for statistic and use this port number [default: 9988].
    CONTAINER                                                       Remote continer name.
    OBJECT                                                          Remote object name.
    PATH                                                            Download object to specific path.
    -o --once                                                       Only run one pass of daemon.
    -u --update                                                     Update to the latest version.
    -h --help                                                       Show this message.
    --version                                                       Show Version.

    service install|uninstall Install/uninstall GoWF service(sysv/systemd/launchctl) script
    service start|stop        Start/stop GoWF service
    config [CONFIG_FILE]      Generante a configuration file


    ** HERE ARE EXPERIMENTAL FEATURES, PLEASE CONTACT SWIFTSTACK FIRST **

    --dedup                   Enable dedup feature for large object.
    --purgededup              Enable purge dedup feature for large object.
    --statdedup               Enable get dedup segment reference counting.

Download latest binary¶

$ curl https://cloud.swiftstack.com/v1/AUTH_gowf/gowf/download-gowf.sh | bash