Home » Blogs » akanugant's blog

Attempt to move dean to new host, associated issues

Updated on Tue, 2025-05-06 09:40. Originally created by akanugant on 2025-05-05 18:27.

Today around 10 am, plan was to migrate dean VM from ovirt2 to ovirt3 (newly added)
For some reason migration failed, but the VM got back to ovirt2 safely with no service interruptions 

Then I saw some vdsm services are missing on ovirt3 and I did try to reboot this host without setting maintenance model ON.
This has to be investigated: ovirt1 engine brought down the ovirt2 (issued a cmd to shutdown) and also the hosted-engine went into a pause mode.

Discovered that ovirt1 and ovirt4 have different checksum (glusterFS) but Wayne manually powered on ovirt2:
ovirt2 now hosts the hosted-engine and also runs the dean VM

Problem now: No redundancy 

Right now, ovirt2 is the only safe host, running the hosted engine and critical VM, and Gluster is degraded, with ovirt1 down and ovirt4 rejected by peers.

Solution 1:

On o1:

systemctl stop glusterd
mv /var/lib/glusterd /var/lib/glusterd.bak.$(date +%s)
mkdir /var/lib/glusterd
systemctl start glusterd

Rejoin from o2:

gluster peer probe ovirt1
gluster peer status

If o1 is connected,

gluster volume status
gluster volume heal ovirt info summary

Then on o2:
gluster peer probe ovirt1
gluster peers status

Heal: gluster volume heal ovirt full

Solution 2: Remove and Re-Add ovirt1 Cleanly
"If ovirt1 doesn’t have any exclusive, unwritten data and you treat ovirt2 as source of truth:"

On o2:
gluster peer detach ovirt4

On o1:

systemctl stop glusterd
mv /var/lib/glusterd /var/lib/glusterd.bak.$(date +%s)
mkdir /var/lib/glusterd
systemctl start glusterd

On o2:

gluster peer probe ovirt1
gluster volume status
gluster volume heal ovirt

  #### Rest of the blogs are now written in Notion #####

 

 

»