Setting up an Amazon ftp server to receive big files

Sharing large files with collaborators has rarely been a problem, we usually just compress them and put them on our web server and then send the link to our collaborator who can then download the file.
However, we have struggled to find a solution to receive large files. We usually run out of space in Dropbox or Google Drive. We have tried infinit.io but this has failed on a few occasions, we think due to firewall issues either on our side or on the side of the collaborator. So when we managed to get an amazon cloud account set-up through Arcus Global (see my previous blog on how we got that organised), an obvious thing to try was to set-up an ftp server to receive large files.

A bit of googling around provided us with a very useful post on Stackoverflow.

First, we launched an instance through the Amazon web interface. We selected an Ubuntu instance from Amazon EC2 and specified 60Gb of storage. We generated a new key called “ftp” and saved the key locally. The .txt extension was added to the file so we renamed it and changed the permissions.

mv ftp.pem.txt ftp.pem
chmod 400 ftp.em

Using the ip address of the instance we then logged into the instance using ssh.

ssh -i ftp.pem ubuntu@12.34.567.89

We installed the ftp server

sudo apt-get install vsftpd
df -h
ps -edf|grep -i ftp
sudo passwd ftp

but we did not provide a password for ftp as we decided to use the ubuntu username and password for login.

Then we opened up the FTP ports on your EC2 instance as described on Stackoverflow.

We changed the ssh configuration as explained in the previous blog and changed the ubuntu user’s password:

sudo vi /etc/ssh/sshd_config
sudo service ssh restart
sudo passwd ubuntu

Then we changed the ftp configuration file in /etc/vsftpd.conf

sudo vi /etc/vsftpd.conf

The following lines where changed:

anonymous_enable=NO

And added the following with the IP of our instance. If restarting an image, the IP will be different so this will need to be changed

pasv_enable=YES
pasv_min_port=1024
pasv_max_port=1048
pasv_address=12.34.567.89

Restart vsftpd

sudo /etc/init.d/vsftpd restart

We had it up for about 24 hours and it cost approximately £0.56.

Categories: amazon, BioLinux, EC2