After moving to 3.5 kernel I haven't seen it, even at 3x traffic which used
to cause it
On Mar 25, 2014 9:06 PM, "Dewey McDonnell" <email address hidden> wrote:

> My web search reveals this problem has been around for years and most
> posts conclude that the problem continues.  After solving my tap0 TX
> packets dropped problem, I find that my VM network has random freezes a
> few times each day.  Many bloggers on this subject say this VM network
> freeze is difficult to reproduce.  Not for me! I can cause a network
> freeze on my VM in a heartbeat.  Like many others, all I need to do is
> start a data transfer over the bridge (FTP 2Kb file or larger). When the
> freeze happens, ifconfig tap0 says overruns:1. This is QEMU version
> 1.6.2 without KVM on CentOS 6.5 version 2.6.32-431.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/997978
>
> Title:
>   KVM images lose connectivity with bridged network
>
> Status in OpenStack Compute (Nova):
>   Invalid
> Status in "qemu-kvm" package in Ubuntu:
>   Fix Released
> Status in "qemu-kvm" source package in Precise:
>   Fix Released
>
> Bug description:
>   =========================================
>   SRU Justification:
>   1. Impact: networking breaks after awhile in kvm guests using virtio
> networking
>   2. Development fix: The bug was fixed upstream and the fix picked up in
> a new
>      merge.
>   3. Stable fix: 3 virtio patches are cherrypicked from upstream:
>      a821ce5 virtio: order index/descriptor reads
>      92045d8 virtio: add missing mb() on enable notification
>      a281ebc virtio: add missing mb() on notification
>   4. Test case: Create a bridge enslaving the real NIC, and use that as
> the bridge
>      for a kvm instance with virtio networking.  See comment #44 for
> specific test
>      case.
>   5. Regression potential: Should be low as several people have tested the
> fixed
>      package under heavy load.
>   =========================================
>
>   System:
>   -----------
>   Dell R410 Dual processor 2.4Ghz w/16G RAM
>   Distributor ID: Ubuntu
>   Description:    Ubuntu 12.04 LTS
>   Release:        12.04
>   Codename:       precise
>
>   Setup:
>   ---------
>   We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged
> networking.
>
>   From the host:
>   # cat /etc/network/interfaces
>   auto br0
>   iface br0 inet static
>           address 212.XX.239.98
>           netmask 255.255.255.240
>           gateway 212.XX.239.97
>           bridge_ports eth0
>           bridge_fd 9
>           bridge_hello 2
>           bridge_maxage 12
>           bridge_stp off
>
>   # ifconfig eth0
>   eth0      Link encap:Ethernet  HWaddr d4:ae:52:84:2d:5a
>             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>             RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0
>             TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0
>             collisions:0 txqueuelen:1000
>             RX bytes:4115980743 (4.1 GB)  TX bytes:5451961979 (5.4 GB)
>             Interrupt:36 Memory:da000000-da012800
>
>   # ifconfig br0
>   br0       Link encap:Ethernet  HWaddr d4:ae:52:84:2d:5a
>             inet addr:212.XX.239.98  Bcast:212.XX.239.111
>  Mask:255.255.255.240
>             inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link
>             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>             RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0
>             TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0
>             collisions:0 txqueuelen:0
>             RX bytes:210152198 (210.1 MB)  TX bytes:300858508 (300.8 MB)
>
>   # brctl show
>   bridge name     bridge id               STP enabled     interfaces
>   br0             8000.d4ae52842d5a       no              eth0
>
>   I have no default network configured to autostart in libvirt as we're
> using bridged networking:
>   # virsh net-list --all
>   Name                 State      Autostart
>   -----------------------------------------
>   default              inactive   no
>
>   # arp
>   Address                  HWtype  HWaddress           Flags Mask
>    Iface
>   mailer03.xxxx.com       ether   52:54:00:82:5f:0f   C
>   br0
>   mailer01.xxxx.com       ether   52:54:00:d2:f7:31   C
>   br0
>   mailer02.xxxx.com       ether   52:54:00:d3:8f:91   C
>   br0
>   dxi-gw2.xxxx.com        ether   00:1a:30:2a:b1:c0   C
>   br0
>
>   From one of the guests:
>   <domain type='kvm' id='4'>
>     <name>mailer01</name>
>     <uuid>d41d1355-84e8-ae23-e84e-227bc0231b97</uuid>
>     <memory>2097152</memory>
>     <currentMemory>2097152</currentMemory>
>     <vcpu>1</vcpu>
>     <os>
>       <type arch='x86_64' machine='pc-1.0'>hvm</type>
>       <boot dev='hd'/>
>     </os>
>     <features>
>       <acpi/>
>     </features>
>     <clock offset='utc'/>
>     <on_poweroff>destroy</on_poweroff>
>     <on_reboot>restart</on_reboot>
>     <on_crash>destroy</on_crash>
>     <devices>
>       <emulator>/usr/bin/kvm</emulator>
>       <disk type='file' device='disk'>
>         <driver name='qemu' type='raw'/>
>         <source file='/dev/mapper/vg_main-mailer01--root'/>
>         <target dev='hda' bus='ide'/>
>         <alias name='ide0-0-0'/>
>         <address type='drive' controller='0' bus='0' unit='0'/>
>       </disk>
>       <disk type='file' device='disk'>
>         <driver name='qemu' type='raw'/>
>         <source file='/dev/mapper/vg_main-mailer01--swap'/>
>         <target dev='hdb' bus='ide'/>
>         <alias name='ide0-0-1'/>
>         <address type='drive' controller='0' bus='0' unit='1'/>
>       </disk>
>       <controller type='ide' index='0'>
>         <alias name='ide0'/>
>         <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
> function='0x1'/>
>       </controller>
>       <interface type='bridge'>
>         <mac address='52:54:00:d2:f7:31'/>
>         <source bridge='br0'/>
>         <target dev='vnet0'/>
>         <model type='virtio'/>
>         <alias name='net0'/>
>         <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
>       </interface>
>       <serial type='pty'>
>         <source path='/dev/pts/0'/>
>         <target port='0'/>
>         <alias name='serial0'/>
>       </serial>
>       <console type='pty' tty='/dev/pts/0'>
>         <source path='/dev/pts/0'/>
>         <target type='serial' port='0'/>
>         <alias name='serial0'/>
>       </console>
>       <input type='mouse' bus='ps2'/>
>       <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'>
>         <listen type='address' address='127.0.0.1'/>
>       </graphics>
>       <video>
>         <model type='cirrus' vram='9216' heads='1'/>
>         <alias name='video0'/>
>         <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>       </video>
>       <memballoon model='virtio'>
>         <alias name='balloon0'/>
>         <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
>       </memballoon>
>     </devices>
>     <seclabel type='dynamic' model='apparmor' relabel='yes'>
>       <label>libvirt-d41d1355-84e8-ae23-e84e-227bc0231b97</label>
>       <imagelabel>libvirt-d41d1355-84e8-ae23-e84e-227bc0231b97</imagelabel>
>     </seclabel>
>   </domain>
>
>   From within the guest:
>   # cat /etc/network/interfaces
>   # The primary network interface
>   auto eth0
>   iface eth0 inet static
>           address 212.XX.239.100
>           netmask 255.255.255.240
>           network 212.XX.239.96
>           broadcast 212.XX.239.111
>           gateway 212.XX.239.97
>
>   # ifconfig
>   eth0      Link encap:Ethernet  HWaddr 52:54:00:d2:f7:31
>             inet addr:212.XX.239.100  Bcast:212.XX.239.111
>  Mask:255.255.255.240
>             inet6 addr: fe80::5054:ff:fed2:f731/64 Scope:Link
>             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>             RX packets:5631830 errors:0 dropped:0 overruns:0 frame:0
>             TX packets:6683416 errors:0 dropped:0 overruns:0 carrier:0
>             collisions:0 txqueuelen:1000
>             RX bytes:2027322829 (2.0 GB)  TX bytes:2076698690 (2.0 GB)
>
>   A commandline which starts the KVM guest:
>   /usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 2048 -smp
> 1,sockets=1,cores=1,threads=1 -name mailer01 -uuid
> d41d1355-84e8-ae23-e84e-227bc0231b97 -nodefconfig -nodefaults -chardev
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/mailer01.monitor,server,nowait
> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
> -drive
> file=/dev/mapper/vg_main-mailer01--root,if=none,id=drive-ide0-0-0,format=raw
> -device
> ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
> -drive
> file=/dev/mapper/vg_main-mailer01--swap,if=none,id=drive-ide0-0-1,format=raw
> -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev
> tap,fd=18,id=hostnet0 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:d2:f7:31,bus=pci.0,addr=0x3
> -chardev pty,id=charserial0 -device
> isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -vga
> cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
>
>   Problem:
>   ------------
>   Periodically (at least once a day), one or more of the guests lose
> network connectivity.  Ping responds with 'host unreachable', even from the
> dom host. Logging in via the serial console shows no problems: eth0 is up,
> can ping the local host, but no outside connectivity.  Restart the network
> (/etc/init.d/networking restart) does nothing. Reboot the machine and it
> comes alive again.
>
>   I've verified there's no arp games going on on the primary host (the
>   arp tables remain the same before - when it had connectivity - and
>   after - when it doesn't.
>
>   This is a critical issue affecting production services on the latest
>   LTS release of Ubuntu. It's similar to an issue which was 'resolved'
>   in 10.04 but appears to have risen its ugly head again.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nova/+bug/997978/+subscriptions
>