Deployment/New Node Checklist: Difference between revisions

From charlesreid1
Line 406: Line 406:
usermod -a -G docker username
usermod -a -G docker username
</pre>
</pre>
==Docker Boats==
Now you're all set up, and it's on to setting up your docker boats.


=All Done=
=All Done=

Revision as of 22:05, 25 March 2017

  • Domain records set up
    • a name base directory to IP of server
    • cname www redirect
    • not scalable, but that's what aws is for.
  • Aptitude
    • apt get update
    • aptitude build scripts
  • Sysadmin stuff
    • Make non-root default user
  • SSH
    • No root login
  • SSL
    • Getting certificate, certifying self as owner of web domain
  • Docker


Domain Records

If you want example.com to point to your new server, log into your host provider and set up the following DNS records.

Set up an A Name record with host @ (this is the A Name equivalent of a wild card) and set its target to the IP address of your server.

Next set up a C Name record with host www (this rule will catch all www.domain.com) and set its target to domain.com. (trailing period is important).

Now, any requests to www.domain.com will be sent to domain.com instead, and any requests to domain.com will go to the IP address of your machine.

Aptitude

Ubuntu 16.04 LTS

Fresh dev machine apt script

Runs apt-get for all the dev things you need. Ubuntu 16.04 LTS. 

#!/bin/sh
#
# Run as root
# 
# Use the -s flag to simulate this command before actually running it,
# as libraries tend to shift around a lot between Ubuntu versions.

echo "export EDITOR=\"vim\"" >> ~/.bash_profile

# Stupid ubuntu packages
# http://askubuntu.com/questions/593433/error-sudo-add-apt-repository-command-not-found#639431
apt-get install software-properties-common

apt-get install -y \
	vim \
	aptitude \
	build-essential \
	checkinstall \
	make \
	m4 \
	bison \
	flex \
	tar \
	perl \
	binutils \
	sed \
	gawk \
	\
	git \
	wget \
	curl \
	docker \
	\
	python2.7 \
	python3 python3-pip \
	\
	libreadline-gplv2-dev  \
	libncursesw5-dev \
	libssl-dev

Dotfiles

Wait until you create a user to install any dotfiles, of course. Root remains plain and uncontaminated.

Unix dotfiles - yargwid repo https://github.com/charlesreid1/yargwid

Mirror: http://git.charlesreid1.com/charlesreid1/yargwid

Users

See Unix/Sysadmin

Add a non-root user

#!/bin/sh

export USERNAME="zappa"

echo "Making user ${USERNAME}"
useradd ${USERNAME}

echo "Setting home directory /home/${USERNAME}"
mkdir -p /home/${USERNAME}
chown ${USERNAME}:${USERNAME} /home/${USERNAME}
usermod -d /home/${USERNAME} ${USERNAME}

echo "Setting ${USERNAME} shell to bash"
usermod -s /bin/bash ${USERNAME}

echo "If you want to add ${USERNAME} to sudo group, run the command yourself:"
echo ""
echo "    usermod -G sudo ${USERNAME}"
echo ""

echo "Set password for ${USERNAME}:"
passwd ${USERNAME}

Once user is in sudo group, no need to add them to sudoers file.

SSH

SSHD Config

Set up the sshd config file, located in /etc/ssh/sshd_config.

Specifically, change PermitRootLogin yes to PermitRootLogin no.

Here's a sed one liner to do it: 

$ sudo sed -i 's/PermitRootLogin yes/PermitRootLogin no/' /etc/sshd_config

then restart the sshd service: 

$ sudo service sshd restart

Or just run this script as sudo: 

#!/bin/sh
sed -i 's/PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config
service sshd restart

SSL

This covers how you get an SSL certificate to run encrypted services on a server.

Lets Encrypt

Getting a certificate for the domain associated with the new node:

Installing Certbot

CertBot directs me to run the following as sudo: 

#!/bin/sh

add-apt-repository ppa:certbot/certbot
apt-get update
apt-get -y install certbot

Obtaining a cert from the "webroot" plugin (of certbot) requires access to the root web directory.

That means, we actually have to have a web server set up. Oh, ok, hang on.

Side note: installing Apache to test certificates

Really quick, let's install an apache web server so we can turn on SSL with it and test out our certificates. 

sudo apt-get install apache2

This installs apache2 to the default location. We can now put a quick "hello world" html file into the web root, which is by default at /var/www/html. (If we don't know where it is, we can check the default apache config file which is in /etc/apache2/, and that will specify the web root location.)

Create a hello world page: 

$ echo "<h2>OHAI WERLD</h2>" > /var/www/html/index.html

May need to be sudo, since by default my /var/www/ was owned by root.

Now start the apache service: 

$ sudo service apache2 restart

Now you can visit your new machine's IP address in the web browser and see "OHAI WERLD".

Note that if you set up any A Name records for a domain to forward to your server, you can also check those now.

This Apache server does not have SSL set up yet, we'll do that once we generate the SSL certificates.

Note: check who the Apache user is by opening /etc/apache2/envvars and check the variable export APACHE_RUN_USER=www-data.

This is the user who should own our web directory /var/www/htdocs: 

$ sudo chown -R www-data:www-data /var/www

Certbot to generate certificates

Now we use Certbot to generate SSL certificates: https://certbot.eff.org/#ubuntuxenial-nginx

Use the cert only framework to generate a web root certificate (the kind you generate if you control the web root directory).

No Arguments: 

$ certbot certonly

This asks for some information, like an email and whether you agree with the Lets Encrypt EULA: https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf

This makes several references to ISRG, which is the Internet Security Research Group, a non-profit based in California that served as the certificate authority willing to sign Lets Encrypt certificates without charging absurd amounts of money: https://letsencrypt.org/isrg/

This asks for your domain names, be sure and enter both example.com and www.example.com.

Well... then the stinker couldn't interpret my web directory input.

With Arguments:

I had more luck with clearly stated error messages when I ran certbot with the various flags to automatically answer the questions above. 

$ sudo certbot certonly --webroot -w /var/www/html -d reidmachine.party -d www.reidmachine.party
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for reidmachine.party
http-01 challenge for wwwreidmachine.party
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. wwwreidmachine.party (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: DNS problem: NXDOMAIN looking up A for wwwreidmachine.party, reidmachine.party (http-01): urn:acme:error:unknownHost :: The server could not resolve a domain name :: No valid IP addresses found for reidmachine.party

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: reidmachine.party
   Type:   unknownHost
   Detail: No valid IP addresses found for reidmachine.party

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address.
 - The following errors were reported by the server:

   Domain: www.reidmachine.party
   Type:   connection
   Detail: DNS problem: NXDOMAIN looking up A for www.reidmachine.party

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

This was a problem with the DNS records (see top of page). The www. domain was not working.

I ran the certificate-making process on just the domain that was working (no prefix) and that worked fine: 

$ sudo certbot certonly --webroot -w /var/www/html -d reidmachine.party
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for reidmachine.party
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0000_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0000_csr-certbot.pem

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/reidmachine.party/fullchain.pem. Your cert
   will expire on 2017-06-23. To obtain a new or tweaked version of
   this certificate in the future, simply run certbot again. To
   non-interactively renew *all* of your certificates, run "certbot
   renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

Once the DNS records were sorted out, and the www. prefix was working, I re-ran the command with both domains. This seamlessly added the new domain to the end of the existing certificate. Pretty slick! 

$ sudo certbot certonly --webroot -w /var/www/html -d reidmachine.party -d www.reidmachine.party
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org

-------------------------------------------------------------------------------
You have an existing certificate that contains a portion of the domains you
requested (ref: /etc/letsencrypt/renewal/reidmachine.party.conf)

It contains these names: reidmachine.party

You requested these names for the new certificate: reidmachine.party,
www.reidmachine.party.

Do you want to expand and replace this existing certificate with the new
certificate?
-------------------------------------------------------------------------------
(E)xpand/(C)ancel: E
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for reidmachine.party
http-01 challenge for www.reidmachine.party
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0001_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0001_csr-certbot.pem

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/reidmachine.party/fullchain.pem. Your cert
   will expire on 2017-06-23. To obtain a new or tweaked version of
   this certificate in the future, simply run certbot again. To
   non-interactively renew *all* of your certificates, run "certbot
   renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

Certbot cron job

Certbot package will run a cron job to renew certificates before expiry. The certificates last for 90 days. To do a dry run of a certificate renewal: 

$ sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/reidmachine.party.conf
-------------------------------------------------------------------------------
Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for reidmachine.party
http-01 challenge for www.reidmachine.party
Waiting for verification...
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0002_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0002_csr-certbot.pem
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/reidmachine.party/fullchain.pem (success)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)

IMPORTANT NOTES:
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.

Docker

At this point, we're ready to install docker, and then load up the new node with whatever containers we need.

Installing

See Docker/Installing 

$ apt-get install docker

Non-Sudo Docker Usage

Modify the user to. 

groupadd docker
usermod -a -G docker username


Docker Boats

Now you're all set up, and it's on to setting up your docker boats.

All Done

Once we've gone from "bare metal" to Docker, we're all set up to build a toolchain for whatever this node happens to be doing.

This means we decide which containers to load into the Docker platform. So we build a boat.

See Docker/Boats for assorted Docker containers that ship together.

Flag

Retrieved from "https://charlesreid1.com/w/index.php?title=Deployment/New_Node_Checklist&oldid=15522"