« «

How to Fix Ceph Error “cluster_uuid file exists with value X, != our uuid Y”

This error can occur if you are trying to perform a recovery from OSDs, and the cluster_uuid extracted from the recovery does not match the monmap.

# Replace 'pve1' with the name of your monitor
# Stop the monitor
systemctl stop ceph-mon@pve1
# Extract the monitor map to a file called monmap
ceph-mon -i pve1 --extract-monmap monmap
# Change FSID
monmaptool --clobber --fsid $NEW_FSID monmap
# Make any other changes via monmaptool, such as rewriting the monitor list
# Inject the new monmap
ceph-mon -i pve1 --inject-monmap monmap
# Start service
systemctl start ceph-mon@pve1


This entry was posted on Saturday, June 8th, 2024 at 08:41 and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

2 Responses to “How to Fix Ceph Error “cluster_uuid file exists with value X, != our uuid Y””

  1. Tiago Matias Says:
    June 16th, 2024 at 04:44

    Hi Matt,

    I am suffering from the same problem recovering an installation of PetaSAN, but it didn’t solve my problem, any additional hints?

    Message on /var/log/ceph/ceph-mon.san-3.log

    2024-06-16T08:40:13.392+0100 7f875ee2da00 0 starting mon.san-3 rank 2 at public addrs [v2:10.0.110.3:3300/0,v1:10.0.110.3:6789/0] at bind addrs [v2:10.0.110.3:3300/0,v1:10.0.110.3:6789/0] mon_data /var/lib/ceph/mon/ceph-san-3 fsid f090a6c1-eb64-45e8-9873-ac109361ebbc
    2024-06-16T08:40:13.396+0100 7f875ee2da00 1 mon.san-3@-1(???) e3 preinit fsid f090a6c1-eb64-45e8-9873-ac109361ebbc
    2024-06-16T08:40:13.396+0100 7f875ee2da00 -1 mon.san-3@-1(???) e3 error: cluster_uuid file exists with value 81acea00-fc7b-4131-a038-43ef1aa2efcc, != our uuid f090a6c1-eb64-45e8-9873-ac109361ebbc
    2024-06-16T08:40:13.396+0100 7f875ee2da00 -1 failed to initialize

    I have done as per your tip….

    systemctl stop ceph-mon
    ceph-mon -i san-3 –extract-monmap /tmp/monmap
    monmaptool –clobber –fsid f090a6c1-eb64-45e8-9873-ac109361ebbc /tmp/monmap
    ceph-mon -i san-3 –inject-monmap /tmp/monmap
    systemctl start ceph-mon

    Keep getting same error :(

    Regards from Portugal,
    Tiago Matias

  2. Matt Ventura Says:
    June 16th, 2024 at 09:19

    Unfortuantely, I don’t have a concrete answer, but can you try `monmaptool –print monmap` after changing the FSID? Make sure it shows the new UUID, and doesn’t contain any incorrect references to the old cluster like monitor IPs that no longer exist.

Leave a Reply