We recently upgraded our Ceph cluster from the latest version of Hammer to 10.2.7 (Jewel). Here are the steps that we used in order to complete the upgrade. Due to a change in Ceph daemon permissions, this specific upgrade required an additional step of using chmod to change file permissions for each daemon directory.<\/p>\n
Set the cluster to the ‘noout’ state so that we can perform the upgrade without any data movement:<\/strong> From the Ceph-deploy control node upgrade monitor nodes first:<\/strong> On each monitor node:<\/strong> Next move on to the OSD nodes:<\/strong> Stop OSD’s and restart them on each node:<\/strong> Don’t forget to unset noout from the admin node:<\/strong> Once the cluster is all healthy again and you have some time make the necessary permission changes for the OSD daemons you can do the following:<\/strong><\/p>\n Set noout:<\/strong> Log onto to each OSD node 1 at a time and run the following commands:<\/strong> Comment out the setuser line in ceph.conf and restart OSD’s:<\/strong> Don’t forget to unset noout from the admin node:<\/strong> We recently upgraded our Ceph cluster from the latest version of Hammer to 10.2.7 (Jewel). Here are the steps that we used in order to complete the upgrade. Due to a change in Ceph daemon permissions, this specific upgrade required an additional step of using chmod to change file permissions for each daemon directory. Set […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[35,3,23],"tags":[],"_links":{"self":[{"href":"https:\/\/www.shainmiley.com\/wordpress\/wp-json\/wp\/v2\/posts\/1655"}],"collection":[{"href":"https:\/\/www.shainmiley.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shainmiley.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shainmiley.com\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shainmiley.com\/wordpress\/wp-json\/wp\/v2\/comments?post=1655"}],"version-history":[{"count":13,"href":"https:\/\/www.shainmiley.com\/wordpress\/wp-json\/wp\/v2\/posts\/1655\/revisions"}],"predecessor-version":[{"id":1694,"href":"https:\/\/www.shainmiley.com\/wordpress\/wp-json\/wp\/v2\/posts\/1655\/revisions\/1694"}],"wp:attachment":[{"href":"https:\/\/www.shainmiley.com\/wordpress\/wp-json\/wp\/v2\/media?parent=1655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shainmiley.com\/wordpress\/wp-json\/wp\/v2\/categories?post=1655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shainmiley.com\/wordpress\/wp-json\/wp\/v2\/tags?post=1655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nceph osd set noout<\/code><\/p>\n
\nceph-deploy install --release jewel ceph-mon1 ceph-mon2 ceph-mon3<\/code><\/p>\n
\nstop ceph-mon-all<\/code>
\ncd \/var\/lib\/ceph<\/code>
\nchown -R ceph:ceph \/var\/lib\/ceph\/<\/code>
\nstart ceph-mon-all<\/code><\/p>\n
\nceph-deploy install --release jewel ceph-osd1 ceph-osd2 ceph-osd3 ceph-osd4<\/code>
\n
\nAdd the following line to \/etc\/ceph\/ceph.conf on each OSD (this will allow the ceph daemons to startup using the old permission scheme):<\/strong>
\nsetuser match path = \/var\/lib\/ceph\/$type\/$cluster-$id<\/code><\/p>\n
\nstop ceph-osd-all<\/code>
\nstart ceph-osd-all<\/code><\/p>\n
\nceph osd unset noout<\/code><\/p>\n
\nceph osd set noout<\/code><\/p>\n
\nfind \/var\/lib\/ceph\/osd -maxdepth 1 -mindepth 1 -print | xargs -P12 -n1 chown -R root:root<\/code><\/p>\nstop ceph-osd-all<\/code><\/p>\nfind \/var\/lib\/ceph\/osd -maxdepth 1 -mindepth 1 -print | xargs -P12 -n1 chown -R ceph:ceph<\/code><\/p>\nchown -R ceph:ceph \/var\/lib\/ceph\/<\/code><\/p>\n
\n#setuser match path = \/var\/lib\/ceph\/$type\/$cluster-$id<\/code>
\nstart ceph-osd-all<\/code><\/p>\n
\nceph osd unset noout<\/code><\/p>\n","protected":false},"excerpt":{"rendered":"