You can use GlusterFS, a NAS (network-attached storage) file system, to setup a shared filesystem across nodes inside Crane. This guide illustrates how to install, setup, and cleanup Gluster. The following guide assumes that the commands are manually executed on each node. Alternatively, Gluster provides this repository for users that prefer to setup distributed systems with Ansible instead.
You can install Gluster on Ubuntu by running the following commands. (Gluster 7 is the latest version as of February 2020)
$ apt install software-properties-common $ add-apt-repository ppa:gluster/glusterfs-7 $ apt update $ apt install glusterfs-server
Start the GlusterFS management daemon and check that it is running.
$ sudo systemctl start glusterd $ systemctl status glusterd
Configure the Trusted Pool¶
Assume we have three nodes with hostnames "server1", "server2" and "server3".
$ sudo gluster peer probe server2 $ sudo gluster peer probe server3
Note: When using hostnames, the first server needs to be probed from one other server to set its hostname.
$ sudo gluster peer probe server1
Note: Once this pool has been established, only trusted members may probe new servers into the pool. A new server cannot probe the pool, it must be probed from the pool.
Check the peer status (on any node):
$ sudo gluster peer status
Create a GlusterFS Volume¶
Create brick directories on each node. (Replace the path to wherever you wish to store data):
$ mkdir -p /data/brick1
Now we will create a Gluster volume named "crane_volume" that distributes files across these bricks. If you wish to configure your volume further (e.g. replicating data), refer to this guide.
From any single node:
$ sudo gluster volume create crane_volume server1:/data/brick1 server2:/data/brick1 server3:/data/brick1 $ sudo gluster volume start
Confirm that the volume's status is "Started":
$ sudo gluster volume info
Mounting the Volume¶
Mount the volume to
/mnt/gluster on each node. For example on server1:
$ sudo mount -t glusterfs server1:crane_volume /mnt/gluster
/mnt/gluster as source directory to use with Docker.
Note: Reusing the stale brick to create a new volume will fail with "
</path/to/brick/> is already part of a volume". Simply put
force at the end of
gluster volume create ..., or execute the following commands.
$ setfattr -x trusted.glusterfs.volume-id (/path/to/brick) $ setfattr -x trusted.gfid (/path/to/brick) $ rm -rf (/path/to/brick)/.glusterfs