Building and Running FiberFS

Github

git clone https://github.com/fiberfs/fiberfs.git

FiberFS on Github

Building

make

FiberFS has 2 optional but recommended package dependencies: openssl and zlib. Don't forget to add "-j4" (4 being the number of available cores to use) to all your make commands.

Testing

make test

Config

At minimum, your FiberFS config file (fiberfs.conf) just needs to define your S3 endpoint:

S3_HOST = my-bucket.s3.region123.cloud-provider.com
S3_REGION = region123
S3_ACCESS_KEY = ACCESS_KEY_STRING
S3_SECRET_KEY = SECRET_KEY_STRING

More configuration parameters are documented below.

Running

./fiberfs [fiberfs.conf] [mount_point]

Logging

./fiberfs_log [mount_point]

S3 Configuration

S3_HOST - Hostname or IP of the S3 endpoint.
S3_REGION - S3 region.
S3_ACCESS_KEY - S3 access key.
S3_SECRET_KEY - S3 secret key.
S3_PREFIX - S3 prefix to use on all operations. Optional.
S3_TLS - Use TLS for S3 communication. Defaults to true.
S3_PORT - S3 port. Defaults to 443 if TLS is enabled, otherwise 80.

FS Configuration

DIRECTORY_TTL_SEC - How long to cache directories before looking for new updates. Defaults to 5 seconds.
MAX_DIRECTORIES - Maximum number of directories to keep in memory at anytime. If a directory is not available in memory, it will be loaded from cache or S3. Defaults to 250. A future feature will introduce a memory based limit (ex: 500MB).
LOG_SIZE - Size of the in-memory log in bytes. Defaults to 128KB.
ASYNC_TASKS - Maximum number of burstable concurrent tasks. After this number is reached, FiberFS will degrade to fair share across all operations. Defaults to 4.

Cache Configuration

CACHE_ROOT - Cache location. Defaults to /tmp/fiberfs_cache. The actual cache root is a hashed subdirectory under this location.
CACHE_SIZE_MB - Cache size in MB. Defaults to 25MB.

Cluster Configuration

CSTORE_SERVER - Set to true to enable the cluster server (cache store). Defaults to false. Cstore logging is placed in a memory log mounted on the cache root.
CSTORE_SERVER_ADDRESS - Network address to listen on.
CSTORE_SERVER_PORT - Network port to listen on.
CSTORE_SERVER_TLS - Enable and only accept TLS for all cluster communication. Note that enabling TLS will disable the ability to splice data off the network which will have a negative performance impact on the cluster. Defaults to false.
CLUSTER - A comma seperated list of ADDRESS:PORT pairs of all the cluster members. Addresses must match the confgured CSTORE_SERVER_ADDRESS for proper operation.
CLUSTER_TLS - Use TLS for cluster communication. Defaults to false.

Note that the cluster server does not need to be enabled on all FiberFS clients, the cluster can be broken up into servers and satellites. Lets say you have 4 hosts with a mounted FiberFS that you want to cluster as servers. Their IPs are 10.0.0.1 thru 10.0.0.4 and each server has 25GB of cache space. Your fiberfs.conf cache and cluster section will look like this:

CACHE_ROOT = /var/cache/fiberfs
CACHE_SIZE_MB = 25000

CSTORE_SERVER = true
CSTORE_SERVER_ADDRESS = 10.0.0.X
CSTORE_SERVER_PORT = 14667

CLUSTER = 10.0.0.1:14667, 10.0.0.2:14667, 10.0.0.3:14667, 10.0.0.4:14667

Replace 10.0.0.X with the actual IP of the host. This 4 hosts cluster will now have 100GB of cache and store copies of all roots, indexes, chunks, and full files. Now lets say we have other hosts which we want to participate in the cluster, but we don't want them to act as servers. These are called satellites. Their fiberfs.conf will look like:

CLUSTER = 10.0.0.1:14667, 10.0.0.2:14667, 10.0.0.3:14667, 10.0.0.4:14667

These satellites will still have a default 25MB cache in /tmp/fiberfs_cache that will locally store roots and indexes only. A copy of all roots, indexes, chunks and full files will be stored on the cluster and available to all servers and satellites.

CDN Configuration

CDN_ENDPOINT - Hostname or IP endpoint for the CDN. If no port is specified, port 443 is used if TLS is enabled, otherwise 80. Multiple endpoints can be specified using a comma seperated list in which case FiberFS will automatically use a hashed traffic distribution.
CDN_TLS - Use TLS for CDN communication. Using TLS prevents spliced IO operations. Defaults to true.

Content

Home