Stunnel/Rsync 443

This page was immensely helpful: http://www.netbits.us/docs/stunnel_rsync.html

This page covers modifications to the instructions on Stunnel/Rsync to run rsync over port 443. This is useful if you need to run rsync over stunnel, and the firewall only allows HTTP or HTTPS traffic.

Server

• Server:
  • Adding secure rsync protocol
  • Setting up rsync server config (which dir to rsync to)
  • Running rsync service
  • Setting up stunnel config
  • Running stunnel service

Rsync

Configure Rsync Server

Run an rsync daemon to listen (locally) for rsync requests (these will come from stunnel).

Use the docker rsync image here: https://git.charlesreid1.com/docker/d-rsync

Set the name of the module. We use "pi". See rsyncd.conf in above repo.

Also set the bind address to be the local host only (since stunnel takes care of the external connection):

allow hosts = 127.0.0.1

Running Rsync Server

Running rsync in a docker container.

Get the docker file, edit the config file, run the build script, then run the run script: https://git.charlesreid1.com/docker/d-rsync

Adding Tcp Wrappers for Secure Rsync Protocol

Rsync is tcp traffic, so define new rsync protocol over port 443.

We do this using /etc/services.

Add the following line to the server's /etc/services (match it exactly):

ssyncd 443/tcp # secure rsync over stunnel 443

(Optional: may need to add this to /etc/hosts.allow but this messed things up for me:

ssyncd : A.B.C.D

where A.B.C.D is the server's IP address.)

See https://git.charlesreid1.com/docker/d-stunnel/src/master/stunnel.server.rsync_over_443.conf for details

Stunnel Server for Rsync

Configure Stunnel Server for Rsync

The rsync daemon will run on port 873. The stunnel connection we will set up will be over port 443. The stunnel configuration file will accept connections on 443 and connect them to port 873.

Note these require the rsync service to be defined following the above steps.

Server stunnel.conf file for rsync over port 443:

output	= /var/log/stunnel4/stunnel.log
cert	= /etc/stunnel/stunnel.fullchain.pem
key		= /etc/stunnel/stunnel.key.pem
pid		= /var/run/stunnel4/stunnel.pid
client	= no

debug = 7
foreground = yes

[rsync]
accept = 443
connect = 873

See https://git.charlesreid1.com/docker/d-stunnel/src/master/stunnel.server.rsync_over_443.conf

Running Stunnel Server for Rsync

Fire up stunnel with the stunnel command:

[remote] $ stunnel

Ideally, nothing will happen. Errors may be printed to the screen, or stunnel may fail silently. Check the log to make sure stunnel is running okay. If you set foreground = yes in the config file, the console will display the contents of the log. Otherwise, check the log via:

[remote] $ tail -f /var/log/stunnel4/stunnel.log

Note that you still won't be able to reach your stunnel server from the outside until you expose the port with iptables.

Opening Server Firewall to Stunnel

Need to open firewall to incoming connections on the stunnel rsync port (273):

[remote] $ iptables -A INPUT -p tcp --dport 273 -j ACCEPT
[remote] $ iptables -A FORWARD -p tcp -j ACCEPT --dport 273 -m state --state NEW

Client

This describes the client configuration required to get rsync working over stunnel.

Rsync Client

The client's rsync will operate, as usual, on local port 873. Behind the scenes, stunnel will transparently map that port to a remote rsync daemon/server, so it will appear to the rsync client that it is communicating with a local rsync daemon/server.

Configuring Rsync Client

No special rsync setup is needed for the client. All we have to do is point the rsync client to the rsync server, and specify one of the modules in our configuration file. The rsync daemon running on the remote server will take care of the rest.

Running Rsync Client

The client does not need to run the rsync daemon, only the server does. When you are on the client, you can just run rsync from the command line, and pass it the location of the rsync server (which is localhost port 873). Stunnel takes care of the rest, so the rsync daemon on the remote server appears to be just another local service.

The rsync server should appear to be a local service on port 873. You can check the local rsync port is being mapped correctly to the remote rsync daemon/server by running:

[client] $ rsync -vv localhost::

This runs rsync and connects to the server at localhost (since no host is specified, it defaults to 873). This is where stunnel is listening and forwarding traffic between the client and server. If the server is listening you should get a simple success message back:

[client] $ rsync -vv localhost::
opening tcp connection to localhost port 873
sending daemon args: --server --sender -vvde.Lsfx . /  (5 args)
pi             	Raspberry Pi Wifi Data

Here is the rsync script that syncs wifi data between Raspberry Pis and the remote server: https://git.charlesreid1.com/rpi/pi-transmission/src/master/rsync_with_server.sh

The rsync_with_server.sh script wraps the basic functionality of running rsync a single time. This script is ultimately called by check_stunnel.sh, which is a "forever loop" that opens the stunnel connection and runs rsync over stunnel to update the server with the latest wifi data from the Raspberry Pi's wifi sensor. We'll cover that script below.

Adding Tcp Wrappers for Secure Rsync Protocol

Add the following to /etc/services on the client:

ssync 273/tcp

(Optional: you may need to add the following to the client /etc/hosts.allow. This actually caused problems for me.)

ssync : LOCAL

See stunnel for rsync config file for Raspberry Pi clients: https://git.charlesreid1.com/rpi/pi-stunnel/src/master/stunnel.rpiclient.rsync_over_273.conf

Stunnel Client

Configuring Stunnel Client for Rsync

Client stunnel over rsync configuration file goes here.

Running Stunnel Client for Rsync

The usual - run stunnel using the stunnel command.

To monitor what is happening use debug = 7 and foreground = yes.

Forever Loop: Listening for Client Changes

To listen for changes to the folder containing wifi data on the Raspberry Pi clients, we want to have a loop that runs forever: it should keep the stunnel connection open if it fails, and it should run rsync over stunnel to keep the latest wifi data (CSV files) up-to-date on the remote server.

It should also be resilient to failure - if the stunnel connection fails (if the Raspberry Pi can't find a wifi network to connect to), the program should not crash but should continue running in the background and attempt to connect after a periodic wait.

This script is contained in check_stunnel.sh in the Raspberry Pi pi-transmission repository: https://git.charlesreid1.com/rpi/pi-transmission/src/master/check_stunnel.sh

Here's that relatively modest script:

#!/bin/bash
#
# Check if stunnel is running. 
# If not, try to start it.
# In any case, try to rsync.
# Then take a nap.

while true; do

	echo "$(ps aux | grep [s]tunnel | grep -v check_stunnel)" 
	if [[ "$(ps aux | grep [s]tunnel | grep -v check_stunnel)" == "" ]]; then
		echo "Stunnel not running, trying to start it..."
		stunnel
		sleep 10
		if [[ "$(ps aux | grep [s]tunnel | grep -v check_stunnel)" == "" ]]; then
			echo "No die!"
		fi
	fi

	# If stunnel is not running, this will raise an error and keep going.
	# Do rsync here
	./rsync_with_server.sh

	# wait X seconds, then do it again.
	sleep 300;

done

Debugging

Debugging problems with the two interacting stunnel-rsync layers can get tricky. Here's a good workflow.

Debugging stunnel

First, if you want to see what stunnel is actually doing, add the following to the stunnel configuration file:

debug = 7
foreground = yes

These will run stunnel in the foreground and print out lots of information. You can run this in a terminal window, then open another window and run rsync commands. You should see activity in the stunnel window, indicating it is initiating a connection with the server and passing traffic.

You can do the same thing on the server to monitor the server instance of stunnel, so if you need to troubleshoot a problem on the server side, edit the server stunnel configuration file and add the debug and foreground options.

Debugging rsync

If you are confident stunnel is working properly and that the problem is with rsync, you can monitor rsync using the system log. rsync does not log to its own log file.

By running tail -f /var/log/syslog on the server in a window, then running rsync over stunnel commands in another, you should see messages about rsync activity showing up in the syslog. This should also give you more helpful and descriptive information when things go wrong, and help you diagnose the error.