Featured image of post Home Lab - Part 6: Setup Qdevice for Proxmox Quorum

Home Lab - Part 6: Setup Qdevice for Proxmox Quorum

Maintain quorum in a two-node Proxmox cluster using a Raspberry Pi as a Qdevice.

Overview

This article explains the steps required to configure a Qdevice within a Proxmox cluster. Typically, a minimum of 3 Proxmox nodes is required in order to maintain quorum. However, using a Qdevice allows for a two-node cluster to operate and maintain quorum, without the need of an entire third Proxmox node.

  • NOTE: This guide is intended for clustered environments with less than three Proxmox nodes, or environments with an even number of nodes where voting can result in a 50/50 situation. If your environment contains three or more Proxmox nodes, configuring a Qdevice is not required

What is Quorum?

Quorum refers to the minimum number of Proxmox nodes that must be online and communicating for the cluster to operate safely and make decisions using a voting system. A cluster requires more than half of its nodes to be online and connected to achieve quorum. This usually requires at least three nodes total, or a higher odd number of nodes.

Proxmox uses Corosync for communication between nodes in a cluster, and quorum is used to avoid split-brain scenarios, where two nodes of a cluster may try to manage the same resources independently, potentially corrupting the data.

The Qdevice Solution

The Corosync QDevice is a service that runs on each Proxmox node, as well as on an additional device (server, raspberry pi etc), which is able to act as a “voting member” within the cluster. For this lab environment, an old Raspberry Pi will be used to provide this service. The Raspberry Pi will be connected to the LAN network where the Proxmox nodes are located.

Requirements

  • An existing Proxmox cluster.
  • Download the Raspberry Pi Imager Tool.
  • SD Card to be used with Raspberry Pi.
  • Additional Ethernet cable to connect RPi to the LAN/Management network.
  • Chosen Qdevice must be running a Debian operating system.

Raspberry Pi Preparation

This section describes the steps required to configure a Raspberry Pi for use as a Qdevice to provide quorum to the Proxmox cluster. Although a Raspberry Pi is chosen for this specific task due to it’s small footprint, other options are available as replacements.

Write OS Image to SD Card

  1. Remove the SD card from the Raspberry Pi and insert into a computer with an available card slot. A USB Card Reader can be used for devices with no built-in SD card slot.
  2. Run the Raspberry Pi Imager application and select the desired Operating System to suit your device model. I opted for the Lite (headless) version, omitting the Desktop environment to reduce resource consumption.
  3. Select the SD card as the Storage method.
  4. An option is available to pre-configure the hostname, default credentials, WiFi network and locate settings if desired. I chose to configure these settings, including enabling SSH with password authentication under the Services tab.
  5. Click Next to begin writing the OS to the SD card.

Optional: DHCP & DNS Configuration (OPNsense)

NOTE: This section is only applicable when using OPNsense for firewalling, routing and DHCP/DNS services. Skip this section if not required.

With the RPi connected to the LAN and powered on, I confirmed that it has received an IP address via DHCP in OPNsense. Clicking on the plus icon under the Command column allows for setting this device to use a DHCP reservation, rather than receiving a random address from the DHCP pool.

Entries for both the Qdevice and the existing Proxmox nodes can be added to DNS (Unbound) for name to IP address resolution. This will allow for us to utilize hostnames when connecting via HTTPS or SSH.

Reboot the RPi to allow it to receive the new IP address from the DHCP reservation. Once back online, confirm that SSH is available to the RPi device on the newly assigned IP address.

1
2
3
4
5
# SSH to the Raspberry Pi using hostname and user provided during install.
ssh linuxadmin@inf-rpi-qdv01

# List assigned IP address for eth0 interface.
ip addr | grep eth0


Qdevice Installation & Setup

Root Account SSH

A requirement for the Qdevice daemon (service) is the ability to SSH into the Qdevice (Raspberry Pi) as the root user account.

  • NOTE: This is not ideal in terms of security, and not recommended in typical scenarios. However, after the initial setup of the Qdevice, firewall rules will be configured in OPNsense to block ingress and egress traffic to this device, other than from the Proxmox nodes, adding a layer of security.
  1. Reset the root user account password.
1
2
# Reset root account password.
sudo passwd root

  1. SSH into the Raspberry Pi (RPi) device using the account created during the installation and SD card imaging process.
  2. Edit the SSH config file /etc/ssh/sshd_config using nano or vi text editors.
1
2
# Edit SSH config file.
sudo nano /etc/ssh/sshd_config
  1. Locate the text PermitRootLogon and set the value to yes.
1
PermitRootLogin yes

  1. Write the changes (CTRL + O), then exit the nano text editor (CTRL + X).
  2. Restart the sshd service.
1
2
# Restart the SSHD service. 
sudo systemctl restart sshd

Package Installation

Packages to be installed:

  • Proxmox Nodes:
    • corosync-qdevice
  • Qdevice:
    • corosync-qnetd
    • corosync-qdevice
  1. SSH into the Raspberry Pi device update the install packages, then install the corosync-qnetd package.
  2. You may be prompted to install some other dependency packages, enter y and press Enter.
1
2
3
4
5
# Update repo list and upgrade installed packages.
sudo apt update && sudo apt upgrade -y

# Install corosync-qnetd package.
sudo apt install corosync-qnetd -y && sudo apt install corosync-qdevice -y


Proxmox Node Configuration

With the Raspberry Pi device now configured, both Proxmox nodes need to be setup to work with the Qdevice.

  1. Login to each Proxmox node (either by SSH or web interface).
  2. Within a shell (terminal) window, install the corosync-qdevice package.
1
2
# Install the Qdevice package. 
apt install corosync-qdevice

  1. Repeat this step for both Proxmox nodes, so the package is installed on both.
  2. Select a Proxmox node and configure the Qdevice connection. Replace
1
2
3
# Configure Qdevice connection.
# Example: pvecm qdevice setup [IP ADDRESS]
pvecm qdevice setup 10.0.10.80
  1. Enter the Raspberry Pi root user account password when prompted.
  2. The Qdevice should now be added to the cluster as a voting member.
  3. Verify using command pvecm status, noting the “QDevice” listed under Membership Information.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2  
Flags:            Quorate Qdevice 

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1    A,V,NMW 10.0.10.100 (local)
0x00000002          1    A,V,NMW 10.0.10.101
0x00000000          1            Qdevice

With the Qdevice now added to the cluster to provide/maintain quorum, if a Proxmox node goes down, the cluster will still be able to function with the Qdevice providing the required vote.


Next Steps

This concludes the steps required to configure a Qdevice used to maintain quorum for Proxmox. The next part in the series will provide the steps required to configure a ZFS storage pool in the Proxmox datacenter, useful when the option for shared storage is not available.


Photo by Jainath Ponnala on Unsplash

All rights reserved.
Built with Hugo
Theme Stack designed by Jimmy