attaching rbd fails because apparmor forbids access to ceph.conf

Hi Paul

Makes sense - I can see why this blocks.

Marking Triaged - this does not really need Confirmation.

This bug was fixed in the package libvirt - 0.9.13-0ubuntu3

---------------
libvirt (0.9.13-0ubuntu3) quantal; urgency=low

  * debian/apparmor/libvirt-qemu: add ceph.conf (LP: #1026404)
  * debian/patches: re-add 9002-better_default_uri_virsh.patch (LP: #1026515)
 -- Serge Hallyn <email address hidden> Thu, 19 Jul 2012 07:58:39 -0500

A proposed test case (for SRU). Paul, does this make sense?

1. apt-get install ceph ceph-common ceph-fs-common ceph-mds qemu-kvm libvirt-bin
2. cat > /etc/ceph/ceph.conf << EOF
[osd]
 osd journal size = 1000
 filestore xattr use omap = true

[mon.a]
 host = localhost
 mon addr = 127.0.0.1:6789

[osd.0]
 host = localhost

[osd.1]
 host = localhost

[mds.a]
 host = localhost

EOF
3. qemu-img create cdboot.img 1G
4. wget http://people.canonical.com/~serge/cdboot-ceph.img
5. virsh define cdboot
6. virsh start cdboot

If ceph.conf could not be read, this will immediately exit with errors including:

unable to find any monitors in conf. please specify monitors via -m monaddr or -c ceph.conf

If ceph.conf could be read, then 'virsh start' will hang for awhile and (though it won't succeed without ceph configured) will NOT show the error

unable to find any monitors in conf. please specify monitors via -m monaddr or -c ceph.conf

(Paul, if you can show in a few steps how to configure a working ceph install that'd be even better)

Below is a mostly paste'n'go Ceph recipe — apt clears the input buffer, so it can't be pasted in its entirety.

I've tested it on a blank VM and it seems to work. It could be a little tidier, e.g. currently logs will end up in /var/log/ceph.

I'm unsure about the test case as you have it above. The failure with Openstack occurs when we try to attach a volume to a running instance, not when the instance is started, so the test case probably needs to simulate that more closely. With a working Ceph handy the test case can now attach a volume and then perhaps poll "virsh dumpxml" for a while to ensure it shows up.

I seem to recall "virsh attach-disk" being a sufficiently incomplete mapping to "virsh attach-device" that it can't be used for RBDs. The test case will probably have to construct a suitable XML fragment and feed it to "virsh attach-device". Something like this should work:

    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source protocol='rbd' name='rbd/blorp'/>
      <target dev='vdz' bus='virtio'/>
    </disk>

# begin recipe

sudo su -

apt-get update && apt-get -y install ceph

# paste from here to end

export CEPH_ROOT_TEMP=$(mktemp -d /tmp/ceph-root.XXXXXXXX)

export MKCEPHFS_TEMP=$(mktemp -d /tmp/mkcephfs.XXXXXXXX)

mkdir /etc/ceph

cat > /etc/ceph/ceph.conf << EOF
[osd]
 osd journal size = 1024
 filestore xattr use omap = true
 osd data = ${CEPH_ROOT_TEMP}/\$name
 osd journal = ${CEPH_ROOT_TEMP}/journal.\$name

[mon.${HOSTNAME}]
 host = ${HOSTNAME}
 mon addr = 127.0.0.1:6789
 mon data = ${CEPH_ROOT_TEMP}/\$name
[osd.0]
 host = ${HOSTNAME}
[osd.1]
 host = ${HOSTNAME}
EOF

mkdir ${CEPH_ROOT_TEMP}/osd.{0,1}

mkcephfs -c /etc/ceph/ceph.conf -d $MKCEPHFS_TEMP --prepare-monmap

mkcephfs -d $MKCEPHFS_TEMP --init-local-daemons osd

mkcephfs -d $MKCEPHFS_TEMP --prepare-mon

mkcephfs -d $MKCEPHFS_TEMP --init-local-daemons mon

/etc/init.d/ceph start

ceph -s

rbd create --size 1024 blorp

rbd info blorp

# end

Hello Paul, or anyone else affected,

Accepted into precise-proposed. The package will build now and be available in a few hours in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Looks good.

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

This bug was fixed in the package libvirt - 0.9.8-2ubuntu17.3

---------------
libvirt (0.9.8-2ubuntu17.3) precise-proposed; urgency=low

  * debian/apparmor/libvirt-qemu: add ceph.conf (LP: #1026404)
 -- Serge Hallyn <email address hidden> Wed, 25 Jul 2012 07:02:47 -0500