Same here, thank you everyone!
-- Kraig Amador
On Friday, November 2, 2012 at 5:42 AM, Jonathan Tullett wrote:
> This bug is considered fixed for me. Not a single network glitch since > installing the package from PPA. Many thanks to the development team! > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/997978 > > Title: > KVM images lose connectivity with bridged network > > Status in OpenStack Compute (Nova): > Invalid > Status in “qemu-kvm” package in Ubuntu: > Fix Released > Status in “qemu-kvm” source package in Precise: > Fix Released > > Bug description: > ========================================= > SRU Justification: > 1. Impact: networking breaks after awhile in kvm guests using virtio networking > 2. Development fix: The bug was fixed upstream and the fix picked up in a new > merge. > 3. Stable fix: 3 virtio patches are cherrypicked from upstream: > a821ce5 virtio: order index/descriptor reads > 92045d8 virtio: add missing mb() on enable notification > a281ebc virtio: add missing mb() on notification > 4. Test case: Create a bridge enslaving the real NIC, and use that as the bridge > for a kvm instance with virtio networking. See comment #44 for specific test > case. > 5. Regression potential: Should be low as several people have tested the fixed > package under heavy load. > ========================================= > > System: > ----------- > Dell R410 Dual processor 2.4Ghz w/16G RAM > Distributor ID: Ubuntu > Description: Ubuntu 12.04 LTS > Release: 12.04 > Codename: precise > > Setup: > --------- > We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. > > From the host: > # cat /etc/network/interfaces > auto br0 > iface br0 inet static > address 212.XX.239.98 > netmask 255.255.255.240 > gateway 212.XX.239.97 > bridge_ports eth0 > bridge_fd 9 > bridge_hello 2 > bridge_maxage 12 > bridge_stp off > > # ifconfig eth0 > eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 > TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) > Interrupt:36 Memory:da000000-da012800 > > # ifconfig br0 > br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a > inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 > inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 > TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) > > # brctl show > bridge name bridge id STP enabled interfaces > br0 8000.d4ae52842d5a no eth0 > > I have no default network configured to autostart in libvirt as we're using bridged networking: > # virsh net-list --all > Name State Autostart > ----------------------------------------- > default inactive no > > # arp > Address HWtype HWaddress Flags Mask Iface > mailer03.xxxx.com (http://mailer03.xxxx.com) ether 52:54:00:82:5f:0f C br0 > mailer01.xxxx.com (http://mailer01.xxxx.com) ether 52:54:00:d2:f7:31 C br0 > mailer02.xxxx.com (http://mailer02.xxxx.com) ether 52:54:00:d3:8f:91 C br0 > dxi-gw2.xxxx.com (http://dxi-gw2.xxxx.com) ether 00:1a:30:2a:b1:c0 C br0 > > From one of the guests: > <domain type='kvm' id='4'> > <name>mailer01</name> > <uuid>d41d1355-84e8-ae23-e84e-227bc0231b97</uuid> > <memory>2097152</memory> > <currentMemory>2097152</currentMemory> > <vcpu>1</vcpu> > <os> > <type arch='x86_64' machine='pc-1.0'>hvm</type> > <boot dev='hd'/> > </os> > <features> > <acpi/> > </features> > <clock offset='utc'/> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>destroy</on_crash> > <devices> > <emulator>/usr/bin/kvm</emulator> > <disk type='file' device='disk'> > <driver name='qemu' type='raw'/> > <source file='/dev/mapper/vg_main-mailer01--root'/> > <target dev='hda' bus='ide'/> > <alias name='ide0-0-0'/> > <address type='drive' controller='0' bus='0' unit='0'/> > </disk> > <disk type='file' device='disk'> > <driver name='qemu' type='raw'/> > <source file='/dev/mapper/vg_main-mailer01--swap'/> > <target dev='hdb' bus='ide'/> > <alias name='ide0-0-1'/> > <address type='drive' controller='0' bus='0' unit='1'/> > </disk> > <controller type='ide' index='0'> > <alias name='ide0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> > </controller> > <interface type='bridge'> > <mac address='52:54:00:d2:f7:31'/> > <source bridge='br0'/> > <target dev='vnet0'/> > <model type='virtio'/> > <alias name='net0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> > </interface> > <serial type='pty'> > <source path='/dev/pts/0'/> > <target port='0'/> > <alias name='serial0'/> > </serial> > <console type='pty' tty='/dev/pts/0'> > <source path='/dev/pts/0'/> > <target type='serial' port='0'/> > <alias name='serial0'/> > </console> > <input type='mouse' bus='ps2'/> > <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'> > <listen type='address' address='127.0.0.1'/> > </graphics> > <video> > <model type='cirrus' vram='9216' heads='1'/> > <alias name='video0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> > </video> > <memballoon model='virtio'> > <alias name='balloon0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> > </memballoon> > </devices> > <seclabel type='dynamic' model='apparmor' relabel='yes'> > <label>libvirt-d41d1355-84e8-ae23-e84e-227bc0231b97</label> > <imagelabel>libvirt-d41d1355-84e8-ae23-e84e-227bc0231b97</imagelabel> > </seclabel> > </domain> > > From within the guest: > # cat /etc/network/interfaces > # The primary network interface > auto eth0 > iface eth0 inet static > address 212.XX.239.100 > netmask 255.255.255.240 > network 212.XX.239.96 > broadcast 212.XX.239.111 > gateway 212.XX.239.97 > > # ifconfig > eth0 Link encap:Ethernet HWaddr 52:54:00:d2:f7:31 > inet addr:212.XX.239.100 Bcast:212.XX.239.111 Mask:255.255.255.240 > inet6 addr: fe80::5054:ff:fed2:f731/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:5631830 errors:0 dropped:0 overruns:0 frame:0 > TX packets:6683416 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:2027322829 (2.0 GB) TX bytes:2076698690 (2.0 GB) > > A commandline which starts the KVM guest: > /usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 2048 -smp 1,sockets=1,cores=1,threads=1 -name mailer01 -uuid d41d1355-84e8-ae23-e84e-227bc0231b97 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/mailer01.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/dev/mapper/vg_main-mailer01--root,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=/dev/mapper/vg_main-mailer01--swap,if=none,id=drive-ide0-0-1,format=raw -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev tap,fd=18,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:d2:f7:31,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 > > Problem: > ------------ > Periodically (at least once a day), one or more of the guests lose network connectivity. Ping responds with 'host unreachable', even from the dom host. Logging in via the serial console shows no problems: eth0 is up, can ping the local host, but no outside connectivity. Restart the network (/etc/init.d/networking restart) does nothing. Reboot the machine and it comes alive again. > > I've verified there's no arp games going on on the primary host (the > arp tables remain the same before - when it had connectivity - and > after - when it doesn't. > > This is a critical issue affecting production services on the latest > LTS release of Ubuntu. It's similar to an issue which was 'resolved' > in 10.04 but appears to have risen its ugly head again. > > To manage notifications about this bug go to: > https://bugs.launchpad.net/nova/+bug/997978/+subscriptions > >
Same here, thank you everyone!
--
Kraig Amador
On Friday, November 2, 2012 at 5:42 AM, Jonathan Tullett wrote:
> This bug is considered fixed for me. Not a single network glitch since /bugs.launchpad .net/bugs/ 997978 ======= ======= ======= ======= ====== ======= ======= ======= ======= ====== interfaces da000000- da012800 XX.239. 111 Mask:255. 255.255. 240 52ff:fe84: 2d5a/64 Scope:Link ------- ------- ------- ------- ------ mailer03. xxxx.com) ether 52:54:00:82:5f:0f C br0 mailer01. xxxx.com) ether 52:54:00:d2:f7:31 C br0 mailer02. xxxx.com) ether 52:54:00:d3:8f:91 C br0 dxi-gw2. xxxx.com) ether 00:1a:30:2a:b1:c0 C br0 /name> 84e8-ae23- e84e-227bc0231b 97</uuid> 2097152< /memory> 2097152< /currentMemory> 'pc-1.0' >hvm</type> destroy< /on_poweroff> restart< /on_reboot> destroy< /on_crash> /usr/bin/ kvm</emulator> dev/mapper/ vg_main- mailer01- -root'/ > dev/mapper/ vg_main- mailer01- -swap'/ > '52:54: 00:d2:f7: 31'/> '127.0. 0.1'/> libvirt- d41d1355- 84e8-ae23- e84e-227bc0231b 97</label> libvirt- d41d1355- 84e8-ae23- e84e-227bc0231b 97</imagelabel> interfaces XX.239. 111 Mask:255. 255.255. 240 ff:fed2: f731/64 Scope:Link 1,cores= 1,threads= 1 -name mailer01 -uuid d41d1355- 84e8-ae23- e84e-227bc0231b 97 -nodefconfig -nodefaults -chardev socket, id=charmonitor, path=/var/ lib/libvirt/ qemu/mailer01. monitor, server, nowait -mon chardev= charmonitor, id=monitor, mode=control -rtc base=utc -no-shutdown -drive file=/dev/ mapper/ vg_main- mailer01- -root,if= none,id= drive-ide0- 0-0,format= raw -device ide-drive, bus=ide. 0,unit= 0,drive= drive-ide0- 0-0,id= ide0-0- 0,bootindex= 1 -drive file=/dev/ mapper/ vg_main- mailer01- -swap,if= none,id= drive-ide0- 0-1,format= raw -device ide-drive, bus=ide. 0,unit= 1,drive= drive-ide0- 0-1,id= ide0-0- 1 -netdev tap,fd= 18,id=hostnet0 -device virtio- net-pci, netdev= hostnet0, id=net0, mac=52: 54:00:d2: f7:31,bus= pci.0,addr= 0x3 -chardev pty,id=charserial0 -device isa-serial, chardev= charserial0, id=serial0 -usb -vnc 127.0.0.1:0 -vga cirrus -device virtio- balloon- pci,id= balloon0, bus=pci. 0,addr= 0x4 d/networking restart) does nothing. Reboot the machine and it comes alive again. /bugs.launchpad .net/nova/ +bug/997978/ +subscriptions
> installing the package from PPA. Many thanks to the development team!
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https:/
>
> Title:
> KVM images lose connectivity with bridged network
>
> Status in OpenStack Compute (Nova):
> Invalid
> Status in “qemu-kvm” package in Ubuntu:
> Fix Released
> Status in “qemu-kvm” source package in Precise:
> Fix Released
>
> Bug description:
> =======
> SRU Justification:
> 1. Impact: networking breaks after awhile in kvm guests using virtio networking
> 2. Development fix: The bug was fixed upstream and the fix picked up in a new
> merge.
> 3. Stable fix: 3 virtio patches are cherrypicked from upstream:
> a821ce5 virtio: order index/descriptor reads
> 92045d8 virtio: add missing mb() on enable notification
> a281ebc virtio: add missing mb() on notification
> 4. Test case: Create a bridge enslaving the real NIC, and use that as the bridge
> for a kvm instance with virtio networking. See comment #44 for specific test
> case.
> 5. Regression potential: Should be low as several people have tested the fixed
> package under heavy load.
> =======
>
> System:
> -----------
> Dell R410 Dual processor 2.4Ghz w/16G RAM
> Distributor ID: Ubuntu
> Description: Ubuntu 12.04 LTS
> Release: 12.04
> Codename: precise
>
> Setup:
> ---------
> We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking.
>
> From the host:
> # cat /etc/network/
> auto br0
> iface br0 inet static
> address 212.XX.239.98
> netmask 255.255.255.240
> gateway 212.XX.239.97
> bridge_ports eth0
> bridge_fd 9
> bridge_hello 2
> bridge_maxage 12
> bridge_stp off
>
> # ifconfig eth0
> eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0
> TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB)
> Interrupt:36 Memory:
>
> # ifconfig br0
> br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a
> inet addr:212.XX.239.98 Bcast:212.
> inet6 addr: fe80::d6ae:
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB)
>
> # brctl show
> bridge name bridge id STP enabled interfaces
> br0 8000.d4ae52842d5a no eth0
>
> I have no default network configured to autostart in libvirt as we're using bridged networking:
> # virsh net-list --all
> Name State Autostart
> -------
> default inactive no
>
> # arp
> Address HWtype HWaddress Flags Mask Iface
> mailer03.xxxx.com (http://
> mailer01.xxxx.com (http://
> mailer02.xxxx.com (http://
> dxi-gw2.xxxx.com (http://
>
> From one of the guests:
> <domain type='kvm' id='4'>
> <name>mailer01<
> <uuid>d41d1355-
> <memory>
> <currentMemory>
> <vcpu>1</vcpu>
> <os>
> <type arch='x86_64' machine=
> <boot dev='hd'/>
> </os>
> <features>
> <acpi/>
> </features>
> <clock offset='utc'/>
> <on_poweroff>
> <on_reboot>
> <on_crash>
> <devices>
> <emulator>
> <disk type='file' device='disk'>
> <driver name='qemu' type='raw'/>
> <source file='/
> <target dev='hda' bus='ide'/>
> <alias name='ide0-0-0'/>
> <address type='drive' controller='0' bus='0' unit='0'/>
> </disk>
> <disk type='file' device='disk'>
> <driver name='qemu' type='raw'/>
> <source file='/
> <target dev='hdb' bus='ide'/>
> <alias name='ide0-0-1'/>
> <address type='drive' controller='0' bus='0' unit='1'/>
> </disk>
> <controller type='ide' index='0'>
> <alias name='ide0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
> </controller>
> <interface type='bridge'>
> <mac address=
> <source bridge='br0'/>
> <target dev='vnet0'/>
> <model type='virtio'/>
> <alias name='net0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
> </interface>
> <serial type='pty'>
> <source path='/dev/pts/0'/>
> <target port='0'/>
> <alias name='serial0'/>
> </serial>
> <console type='pty' tty='/dev/pts/0'>
> <source path='/dev/pts/0'/>
> <target type='serial' port='0'/>
> <alias name='serial0'/>
> </console>
> <input type='mouse' bus='ps2'/>
> <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'>
> <listen type='address' address=
> </graphics>
> <video>
> <model type='cirrus' vram='9216' heads='1'/>
> <alias name='video0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
> </video>
> <memballoon model='virtio'>
> <alias name='balloon0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
> </memballoon>
> </devices>
> <seclabel type='dynamic' model='apparmor' relabel='yes'>
> <label>
> <imagelabel>
> </seclabel>
> </domain>
>
> From within the guest:
> # cat /etc/network/
> # The primary network interface
> auto eth0
> iface eth0 inet static
> address 212.XX.239.100
> netmask 255.255.255.240
> network 212.XX.239.96
> broadcast 212.XX.239.111
> gateway 212.XX.239.97
>
> # ifconfig
> eth0 Link encap:Ethernet HWaddr 52:54:00:d2:f7:31
> inet addr:212.XX.239.100 Bcast:212.
> inet6 addr: fe80::5054:
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:5631830 errors:0 dropped:0 overruns:0 frame:0
> TX packets:6683416 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:2027322829 (2.0 GB) TX bytes:2076698690 (2.0 GB)
>
> A commandline which starts the KVM guest:
> /usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 2048 -smp 1,sockets=
>
> Problem:
> ------------
> Periodically (at least once a day), one or more of the guests lose network connectivity. Ping responds with 'host unreachable', even from the dom host. Logging in via the serial console shows no problems: eth0 is up, can ping the local host, but no outside connectivity. Restart the network (/etc/init.
>
> I've verified there's no arp games going on on the primary host (the
> arp tables remain the same before - when it had connectivity - and
> after - when it doesn't.
>
> This is a critical issue affecting production services on the latest
> LTS release of Ubuntu. It's similar to an issue which was 'resolved'
> in 10.04 but appears to have risen its ugly head again.
>
> To manage notifications about this bug go to:
> https:/
>
>