This page was immensely helpful: http://www.netbits.us/docs/stunnel_rsync.html
NOTE: I was not able to get this working reliably on port 273, but was able to get stunnel+rsync working reliably over port 443. Mileage may vary.
- 1 Overview/Approach
- 2 Server
- 3 Client
- 4 Debugging
- 5 Flags
Used the following approach:
- Reorganized all of the files in the repository so that each service had its own configuration and build sequence
- Makefile was entry point: make rsync or make ssh
- The Makefile would set the port, and feed that as an argument to each script.
- Build script, run script, open firewall script
- This enabled flexibility in port number, and enabled switching from 273 to 443 more easily
- Better organization of stunnel - since each service requires different configurations and ports should be easy to switch in and out.
- Adding secure rsync protocol
- Setting up rsync server config (which dir to rsync to)
- Running rsync service
- Setting up stunnel config
- Running stunnel service
Configure Rsync Server
Main repository: https://git.charlesreid1.com/docker/d-stunnel
To configure the rsync stunnel server:
- First, define a universal rsync stunnel configuration file.
- Everywhere you need to specify the numerical port, use the token PORT
- The Makefile will ultimately set the port number, and pass that into the build script. The build script will use sed to find/replace the token PORT with the numerical port.
Makefile that calls build and run scripts: https://git.charlesreid1.com/docker/d-stunnel/src/master/Makefile
Build rsync stunnel docker container script: https://git.charlesreid1.com/docker/d-stunnel/src/master/build_rsync.sh
The build docker container script actually uses the Dockerfile, which assembles the various files that are needed to run both stunnel and rsync. The build script updates the port number in both the Dockerfile and in the rsync stunnel configuration file using sed, then bakes these files into the final container image using docker build.
Universal rsync stunnel configuration file: https://git.charlesreid1.com/docker/d-stunnel/src/master/stunnel.server.rsync.conf
Finally, the Makefile calls the run script and passes the port number to the run script, which hooks up the necessary ports between the host and the container, mounts the necessary volumes, and sets up the docker network interface (as well as running the container in the foreground or background).
Run rsync stunnel container: https://git.charlesreid1.com/docker/d-stunnel/src/master/run_rsync.sh
Running Rsync Server
Once you have set the rsync configuration file, the rsync daemon will run like any standard system service.
$ sudo service rsync start
Alternatively, you can get an rsync docker container up and running: Docker/Rsync
Adding Tcp Wrappers for Secure Rsync Protocol
The next thing we need to do is give the system a bit more information about the protocol we are using.
Rsync is just tcp traffic, so we can define a new service and tell the system what port and protocol it uses.
We do this using
Add the following line to the server's
/etc/services (match it exactly):
ssyncd 273/tcp # secure rsync over stunnel
(Optional: may need to add this to
/etc/hosts.allow but this messed things up for me:
ssyncd : A.B.C.D
where A.B.C.D is the server's IP address.)
Stunnel Server for Rsync
Configure Stunnel Server for Rsync
The rsync daemon will run on port 873. The stunnel connection we will set up will be over port 273. The stunnel configuration file will accept connections on 273 and connect them to port 873.
Note these require the rsync service to be defined following the above steps.
Server stunnel.conf file for rsync over port 273:
output = /var/log/stunnel4/stunnel.log cert = /etc/stunnel/stunnel.fullchain.pem key = /etc/stunnel/stunnel.key.pem pid = /var/run/stunnel4/stunnel.pid client = no debug = 7 foreground = yes [rsync] accept = 273 connect = 873
Running Stunnel Server for Rsync
Fire up stunnel with the
[remote] $ stunnel
Ideally, nothing will happen. Errors may be printed to the screen, or stunnel may fail silently. Check the log to make sure stunnel is running okay. If you set
foreground = yes in the config file, the console will display the contents of the log. Otherwise, check the log via:
[remote] $ tail -f /var/log/stunnel4/stunnel.log
Note that you still won't be able to reach your stunnel server from the outside until you expose the port with iptables.
Opening Server Firewall to Stunnel
Need to open firewall to incoming connections on the stunnel rsync port (273):
[remote] $ iptables -A INPUT -p tcp --dport 273 -j ACCEPT [remote] $ iptables -A FORWARD -p tcp -j ACCEPT --dport 273 -m state --state NEW
This describes the client configuration required to get rsync working over stunnel.
The client's rsync will operate, as usual, on local port 873. Behind the scenes, stunnel will transparently map that port to a remote rsync daemon/server, so it will appear to the rsync client that it is communicating with a local rsync daemon/server.
Configuring Rsync Client
No special rsync setup is needed for the client. All we have to do is point the rsync client to the rsync server, and specify one of the modules in our configuration file. The rsync daemon running on the remote server will take care of the rest.
Running Rsync Client
The client does not need to run the rsync daemon, only the server does. When you are on the client, you can just run rsync from the command line, and pass it the location of the rsync server (which is localhost port 873). Stunnel takes care of the rest, so the rsync daemon on the remote server appears to be just another local service.
The rsync server should appear to be a local service on port 873. You can check the local rsync port is being mapped correctly to the remote rsync daemon/server by running:
[client] $ rsync -vv localhost::
This runs rsync and connects to the server at localhost (since no host is specified, it defaults to 873). This is where stunnel is listening and forwarding traffic between the client and server. If the server is listening you should get a simple success message back:
[client] $ rsync -vv localhost:: opening tcp connection to localhost port 873 sending daemon args: --server --sender -vvde.Lsfx . / (5 args) pi Raspberry Pi Wifi Data
Here is the rsync script that syncs wifi data between Raspberry Pis and the remote server: https://git.charlesreid1.com/rpi/pi-transmission/src/master/rsync_with_server.sh
rsync_with_server.sh script wraps the basic functionality of running rsync a single time. This script is ultimately called by
check_stunnel.sh, which is a "forever loop" that opens the stunnel connection and runs rsync over stunnel to update the server with the latest wifi data from the Raspberry Pi's wifi sensor. We'll cover that script below.
Adding Tcp Wrappers for Secure Rsync Protocol
Add the following to
/etc/services on the client:
(Optional: you may need to add the following to the client
/etc/hosts.allow. This actually caused problems for me.)
ssync : LOCAL
See stunnel for rsync config file for Raspberry Pi clients: https://git.charlesreid1.com/rpi/pi-stunnel/src/master/stunnel.rpiclient.rsync_over_273.conf
Configuring Stunnel Client for Rsync
Client stunnel over rsync configuration file goes here.
Running Stunnel Client for Rsync
The usual - run stunnel using the stunnel command.
To monitor what is happening use debug = 7 and foreground = yes.
Forever Loop: Listening for Client Changes
To listen for changes to the folder containing wifi data on the Raspberry Pi clients, we want to have a loop that runs forever: it should keep the stunnel connection open if it fails, and it should run rsync over stunnel to keep the latest wifi data (CSV files) up-to-date on the remote server.
It should also be resilient to failure - if the stunnel connection fails (if the Raspberry Pi can't find a wifi network to connect to), the program should not crash but should continue running in the background and attempt to connect after a periodic wait.
This script is contained in
check_stunnel.sh in the Raspberry Pi pi-transmission repository: https://git.charlesreid1.com/rpi/pi-transmission/src/master/check_stunnel.sh
Here's that relatively modest script:
#!/bin/bash # # Check if stunnel is running. # If not, try to start it. # In any case, try to rsync. # Then take a nap. while true; do echo "$(ps aux | grep [s]tunnel | grep -v check_stunnel)" if [[ "$(ps aux | grep [s]tunnel | grep -v check_stunnel)" == "" ]]; then echo "Stunnel not running, trying to start it..." stunnel sleep 10 if [[ "$(ps aux | grep [s]tunnel | grep -v check_stunnel)" == "" ]]; then echo "No die!" fi fi # If stunnel is not running, this will raise an error and keep going. # Do rsync here ./rsync_with_server.sh # wait X seconds, then do it again. sleep 300; done
Debugging problems with the two interacting stunnel-rsync layers can get tricky. Here's a good workflow.
First, if you want to see what stunnel is actually doing, add the following to the stunnel configuration file:
debug = 7 foreground = yes
These will run stunnel in the foreground and print out lots of information. You can run this in a terminal window, then open another window and run rsync commands. You should see activity in the stunnel window, indicating it is initiating a connection with the server and passing traffic.
You can do the same thing on the server to monitor the server instance of stunnel, so if you need to troubleshoot a problem on the server side, edit the server stunnel configuration file and add the debug and foreground options.
If you are confident stunnel is working properly and that the problem is with rsync, you can monitor rsync using the system log. rsync does not log to its own log file.
tail -f /var/log/syslog on the server in a window, then running rsync over stunnel commands in another, you should see messages about rsync activity showing up in the syslog. This should also give you more helpful and descriptive information when things go wrong, and help you diagnose the error.
stunnelsecure tunnel - create secure encrypted connections on any port to wrap any protocol
Stunnel Over Docker: Stunnel/Docker
Flags · Template:StunnelFlag · e