Static sites on AWS with S3, CloudFront, Route 53, and Certificate Manager

I primarily use this domain to map subdomains to IP addresses, but AWS makes it too simple to not post a simple static site. The hardest part is navigating the notoriously poorly-named AWS services.

Here, I outline the steps necessary to get a serverless static site set up on AWS.

Step 1: Storing static content on S3

  • Create an S3 bucket (eg “mattswebsite”). If you leave all the default settings, it should create a private bucket.
  • Upload your website (or a placeholder index.html) via the web interface or AWS CLI.
    aws s3 cp website-dev s3://mattswebsite --recursive

Step 2: Managing DNS records with Route 53

Since I puchased my domain from Namecheap, I’ll need to point the domain to Amazon nameservers.

  • ‘Create Hosted Zone’. Make a note of the NS (nameserver) record values.
  • On Namecheap > Advanced DNS, change DNS type to ‘Custom DNS’ and enter your Route 53 NS records.

Step 3: Enabling SSL encryption with Certificate Manager

  • ‘Request a certificate’
  • Validate ownership of the domain by email or DNS record. Since your DNS records are managed by Route 53, you can simply click ‘Create record in Route 53’ and AWS will create the required CNAME records.

Step 4: Caching your static content with CloudFront

  • ‘Create distribution’ > Web (Get Started)

Under ‘Origin Settings’

  • Select your S3 bucket for ‘Origin Domain Name’
  • ‘Rescrict Bucket Access’ to Yes
  • ‘Origin Access Identity’ to ‘Create a New Identity’
  • ‘Grant Read Permissions on Bucket’ to ‘Yes, Update Bucket Policy’

Under ‘Default Cache Behavior Settings’:

  • Change ‘Viewer Protocol Policy’ to ‘Redirect HTTP to HTTPS’
  • ‘Compress Objects Automatically’ to ‘Yes’

Under ‘Distribution Settings’

  • SSL Certificate > Check ‘Custom SSL’. In the text box, select your new ACM certificate.
  • ‘Default Root Object’ to ‘index.html’
  • In the ‘Alternate Domain Names (CNAMEs)’ text box, enter your domain names (eg mattdeboth.com, www.mattdeboth.com)

Step 5: Pointing Route 53 to the CloudFront distribution

  • ‘Create Record Set’
  • Create an A record for your domain and any subdomains (eg mattdeboth.com and www.mattdeboth.com). If you click ‘Alias’: ‘Yes’, scroll down and select your CloudFront distribution.

Adding SCP/SFTP to curl & RCurl on Mac OS X

Update: As of 2019-01-17, Homebrew curl no longer supports the –with-libssh2 option. Instead, curl needs to be downloaded and compiled from source with the flag. The instructions below are incomplete.


RCurl provides wrapper R functions for most protocols supported by libcurl. I like to use the RCurl::scp() function to read files from remote servers, but every time I try to use it on a new Mac, I’m reminded that the OS X system curl is not built with SCP/SFTP support, and therefore, the corresponding R functions won’t work.

To get RCurl working with SCP/SFTP, we first need to install a new version of curl built with libssh2.


1. Install Homebrew

If you’re not already using Homebrew on your Mac, I recommend checking it out. It makes installing/removing software super simple.

You should probably go to the Homebrew site for installation instructions, but if you want to take my word for it, you can install Homebrew like this…

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

2. Install libssh2 and curl

brew install libssh2
brew install curl --with-libssh2

3. Install RCurl with the new curl

Outside of RCurl, I don’t really need a new version of curl with scp/sftp support, and overwriting the system curl is probably a bad idea anyways. Instead, when we install RCurl, you can temporarily add the new curl to your $PATH.

Sys.setenv(PATH=paste('/usr/local/opt/curl/bin', Sys.getenv('PATH'), sep=":"))
install.packages("RCurl", type="source")

Finally, you can make sure it worked by checking the available protocols…

> library(RCurl)
> curlVersion()$protocols
 [1] "dict"   "file"   "ftp"    "ftps"   "gopher" "http"   "https"  "imap"
 [9] "imaps"  "ldap"   "ldaps"  "pop3"   "pop3s"  "rtsp"   "scp"    "sftp"
[17] "smb"    "smbs"   "smtp"   "smtps"  "telnet" "tftp"

Essential Linux commands for data analysis

ssh
connect to a Unix server
ssh username@hostname

pwd
prints the current working directory
pwd

ls
list directory contents
ls
ls ../another/directory

cd
change directory
cd ../ Move up one directory.
cd ~ Return to home directory.
cd /path/to/move/to

Reading files

cat/zcat
print a file to screen

cat myfile.txt

less
print a file, paginated

less myfile.txt

head/tail
print the first/last n lines

head -15 myfile.txt

Manipulating files & directories

nano1
basic text editor

mkdir

mkdir new_folder

create nested directories
mkdir -p new_folder1/new_folder2

cp
cp file_to_copy.txt copy_file_name.txt

mv
move (or rename) a file
mv resum.txt resume.txt

rm
delete stuff

delete a single file
rm myfile.txt

recursively delete a directory
rm mydirectory

Manipulating data

cut
slice a column from a delimited file

print the second column from a comma-delimited file
cut -d, -f2

wc
count characters, words, or lines

count lines in myfile.txt
wc -l myfile.txt

sort sort a file

sort file in reverse numerical order of the fourth column
sort -nrk 4

Misc commands

find

grep

wget

gzip/gunzip

tar


  1. I prefer Vim, but the learning curve is very steep. [return]