9. AWS Elastic File System
Integrating Autoscaling with Amazon Elastic File System
You may want to mount an EFS drive if you are recording files, instead of using Amazon S3 for storage. This option is a little cleaner, as you can bypass the post-process step involved in using S3 storage. The only limitation currently is that EFS is VPC-based, so you would need a different EFS created for each region.
Access the EFS Dashboard here, and click on the Create file system button.
- Configure file system access: Select the VPC that you are using for your autoscaling nodegroups from the dropdown list.
- Create mount targets: The subnets that you created should be automatically selected. Under
security groupslocate the policy that you set up for your nodes and add that. Click on Next Step
- Configure optional settings: at this time we suggest keeping the defaults (General Purpose performance mode and no encryption added)
- Review and create: click on Create File System after reviewing the values.
- Make note of the
File system IDand
DNS nameof your EFS. You will need this information for connection.
It is necessary to keep the following in mind while creating a new
- EFS is VPC based, so make sure to select the correct VPC (the one you are going to use for autoscaling).
- The VPC should have a subnet in each availability zone and each subnet should be selected as a
mount targetfor the File System. This will ensure that the EFS will be mountable in any of the availability zones for that region.
- Ensure that the security group for the EFS is properly configured to allow access from EC2 instances.
- In the
Configure optional settingsscreen, select General Purpose (default) as the performance mode.
- Do not select
Enable encryption of data at rest.
General Settings for EFS Security Group
- Inbound: Add a
Custom TCP Rulefor port
2049allowing NFS read/write. For a simple but less secure configuration use
Source. If you know the addressing scheme and IP ranges for your VPC you can try specifying that was well.
- Outbound: Allow all traffic for all port ranges and any
sudo apt-get install nfs-common
Once you have installed the NFS client and your EFS is configured with proper access security, you should test the connection.
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport <File-System-DNS-Name>:/ <Mount-Destination-Path>
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-de56fa55.efs.us-east-2.amazonaws.com:/ /usr/local/red5pro/webapps/live/streams
To verify that the filesystem is mounted, run
df -h and you should see the path listed at the bottom, for example:
Filesystem Size Used Avail Use% Mounted on udev 1.9G 0 1.9G 0% /dev tmpfs 372M 5.2M 367M 2% /run /dev/nvme0n1p1 16G 1.5G 14G 10% / tmpfs 1.9G 0 1.9G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup tmpfs 372M 0 372M 0% /run/user/1000 fs-de56fa55.efs.us-east-2.amazonaws.com:/ 8.0E 0 8.0E 0% /usr/local/red5pro/webapps/live/streams
You will want to create a shell script to run the mount command, and then add a Crontab job to mount on reboot.
nfsmount.sh added to the ubuntu home directory:
#! /bin/bash sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-de56fa55.efs.us-east-2.amazonaws.com:/ /usr/local/red5pro/webapps/live/streams
To create a crontab entry,
crontab -e and select your preferred editor. Add the following:
# m h dom mon dow command @reboot /home/ubuntu/nfsmount.sh
- To verify your script/crontab, restart the VM and run
df -hto check that the nfs mount is included. Then create an AMI from this VM.