After some trial and (mostly) error (using NFS), I got the backups from xcp-ng working against my Jottacloud storage. So for documentation purposes, I’ll start over (and also fix some of the mistakes I made with the settings, which stored the files in the wrong place).
Requirements
This is written for the use of:
* xcp-ng (Xen Orchestra) for my virtual servers that needs to be backed up
* Jottacloud account (or another one that uses the Jotta backend) for storage
* local server acting as the client for Jottacloud and as the S3 server which Xen Orchestra/xcp-ng will connect to. This can be a virtual one or a separate machine, and it can be used for anything else (I use the virtual server which is also my Xen Orchestra host).
* rclone on the local server as both a client and server, to handle what’s coming in through S3 and push it to Jotta
It should be no problem adopting it to other configurations, but this is what I have to test on.
The local server
As I decided to use the host for Xen Orchestra, it was to begin with a little underpowered (even disk space was low, because I went cheap on it in the beginning), I had to increase disk space and extend the root file system first (I will not describe how, but doing it afterwards involves taking the swap partition down and remove it to be able to expand the file system). I also increased the RAM from 2GB to 4GB and CPU count from 2 to 16 to give it the performance whenever needed.
Installing rclone is straightforward on Linux systems, so just follow the single-line instruction in the documentation:
sudo -v ; curl https://rclone.org/install.sh | sudo bash
Configuring a Jottacloud remote for rclone
After the installation it’s time to configure a remote. The ‘remote’ is in this case the Jottacloud service. By configuring rclone, you will set up a ‘device’ and a ‘mountpoint’ on the Jotta storage.
To compare with the Jotta GUI client, the ‘device’ is the computer being backed up, and ‘mountpoint’ is the folder to be backed up (that is one of the entries listed in the Jotta client main window).
The specifics for each service can be found on its own page in the documentation:
Jottacloud configuration for rclone
A few thing to be mentioned about the whitelabel variants of Jottacloud is that many of them requires you to select “Legacy authentication” (if you do not have the option to use Jotta-CLI and generate a login token)
You may also have to select some non-default replies in the configuration guide when you run the rclone config
command. The differences I had to make compared to the description in the documentation is marked in red in the session below (my text input is marked with bold on prompts marked in red):
No remotes found, make a new one? n) New remote s) Set configuration password q) Quit config n/s/q> n name> s3 Option Storage. Type of storage to configure. Choose a number from below, or type in your own value. [snip] XX / Jottacloud \ (jottacloud) [snip] Storage> 28 (this is currently the number in the list) Option client_id. OAuth Client Id. Leave blank normally. Enter a value. Press Enter to leave empty. client_id> Option client_secret. OAuth Client Secret. Leave blank normally. Enter a value. Press Enter to leave empty. client_secret> Edit advanced config? y) Yes n) No (default) y/n> Option config_type. Select authentication type. Choose a number from below, or type in an existing value of type string. Press Enter for the default (standard). / Standard authentication. 1 | Use this if you're a normal Jottacloud user. \ (standard) / Legacy authentication. 2 | This is only required for certain whitelabel versions of Jottacloud and not recommended for normal users. \ (legacy) / Telia Cloud authentication. 3 | Use this if you are using Telia Cloud (Sweden). \ (telia_se) / Telia Sky authentication. 4 | Use this if you are using Telia Sky (Norway). \ (telia_no) / Tele2 Cloud authentication. 5 | Use this if you are using Tele2 Cloud. \ (tele2) / Onlime Cloud authentication. 6 | Use this if you are using Onlime Cloud. \ (onlime) config_type> 2 Do you want to create a machine specific API key? Rclone has it's own Jottacloud API KEY which works fine as long as one only uses rclone on a single machine. When you want to use rclone with this account on more than one machine it's recommended to create a machine specific API key. These keys can NOT be shared between machines. y) Yes n) No (default) y/n> y Option config_username. Username (e-mail address) Enter a value. config_username> yourjottaemail@fake.com Option config_password. Password (only used in setup, will not be stored) Choose an alternative below. Press Enter for the default (n). y) Yes, type in my own password g) Generate random password n) No, leave this optional password blank (default) y/g/n> y Enter the password: password: Confirm the password: password: Use a non-standard device/mountpoint? Choosing no, the default, will let you access the storage used for the archive section of the official Jottacloud client. If you instead want to access the sync or the backup section, for example, you must choose yes. y) Yes n) No (default) y/n> y Option config_device. The device to use. In standard setup the built-in Jotta device is used, which contains predefined mountpoints for archive, sync etc. All other devices are treated as backup devices by the official Jottacloud client. You may create a new by entering a unique name. Choose a number from below, or type in your own value of type string. Press Enter for the default (Jotta). 1 > Jotta 2 > your other 3 > devices configured 4 > from the client config_device> deb12-xo Option config_mountpoint. The mountpoint to use on the non-standard device deb12-xo. You may create a new by entering a unique name. Choose a number from below, or type in your own value of type string. Press Enter for the default (xcp-ng). 1 > xcp-ng 2 > xcpng-s3 config_mountpoint> rclone-s3 Configuration complete. Options: - type: jottacloud - configVersion: 0 - client_id: yourownstringofcharacters - client_secret: thisissecretsoIwouldnotshowit - username: yourjottaemail@fake.com - password: - auth_code: - token: {"access_token":"supersecretstuffheredonotshare","expiry":"2025-01-28T16:12:46.077725923+01:00"} - device: deb12-xo - mountpoint: rclone-s3 Keep this "s3" remote? y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y Current remotes: Name Type ==== ==== s3 jottacloud e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> q
After the configuration step, you have to connect to it for it to be created online. This is also needed because the ‘bucket’ (folder) has to be created as explained in the rclone serve s3 documentation.
To create the device (deb12-xo above) and mountpoint (rclone-s3 above), mount the remote using the rclone mount
command:
mkdir s3 rclone -vv mount s3: /home/xo/s3
(you will see a warning that it is recommended to use –vfs-cache-mode for the remote, but it’s safe to ignore it in this step)
In another shell, go to the directory where you mounted the remote. Verify that the remote is mounted with df -h .
Create a folder with the name that will be your xcp-ng backup location (“bucket name”). This will create the directories (mountpoint and bucket) on the online storage. If this is not done (or just if you want), check the logs in the other shell.
In this step, I created the directory “xcp-ng” inside my S3 folder.
Now everything is prepared for configuring it within xcp-ng.
Jump out of the s3 directory (just “cd”), then break the connection and mount with ctrl-c in the other shell.
Start the s3 server (temporarily for testing):
rclone -vv serve s3 --auth-key jottatest,sUperSecret --addr=0.0.0.0:8080 s3:
Verify that you can connect using a S3 client
This is an optional step, but you might find it useful now and later.
To prevent possible mistakes when directly testing with xcp-ng, you can download some S3 client to test with first. I found S3 Browser useful enough for testing.
Account setup in this client is simple:
Display name: whatever you want
Account type: S3 Compatible Storage
API endpoint: ip address and port of machine running rclone serve command, as usual like: 10.0.0.222:8080
Access KeyID: the user name (“jottatest” above)
Secret Access Key: the password (“sUperSecret” above)
Check that uploading files works by verifying that they becomes visible online.
The path (visible, not URL) to the files at Jottacloud would be (from the Web UI):
Backups > deb12-xo > rclone-s3 > xcp-ng
Setting up S3 storage in Xen Orchestra
To set up the newly created S3 storage in Xen Orchestra, go to “Settings/Remotes”.
Set type to “Amazon Web Services S3”
Disable HTTPS (for now, the rclone server supports it, but I haven’t tested it)
AWS S3 endpoint: IP address and port of machine running rclone serve s3
AWS S3 bucket name: xcp-ng
Directory: / (blank would currently not be accepted by Xen Orchestra)
Access key ID: the user name, “jottatest”
Secret (field below): the password (“sUperSecret”)
If the test above went well, the connection in Orchestra will just go through and will be enabled and speed-tested.
Now the setup is ready for the first tests with backing up VMs through the connection. If all goes well, make the rclone command run at the start of the computer or VM it’s on. Any way you like.
It can be told to run in the background with the –daemon option. All information you need is in the documentation or in the forums.