Error when copying directory tree with Nautilus to ~/Private using ecryptfs

This same problem occurs when using `cp` on the command line in gnome-terminal:

$ cp -r foo /home/me/Private/
cp: cannot create regular file `/home/me/Private/foo/putty private ssh key.ppk': Cannot allocate memory

I can report the same problem (error message: "Cannot allocate memory") when using the method outlined in
https://help.ubuntu.com/community/EncryptedPrivateDirectory

# Move the application's data directory (e.g. ~/.mozilla or ~/.evolution) into your ~/Private directory

 mv ~/.evolution ~/Private

Hmm, I'm subscribing Mike Halcrow, as this could potentially be a kernel problem.

Mike?

:-Dustin

For those affected by this problem, can you please confirm what kernel you're running?

 $ uname -a

:-Dustin

Hi Dustin,
thanks for looking into this.
I have since deleted the setup that gave me the problem and returned to a fresh install of Xubuntu 8.10 (without ecryptfs obviously), so I can't give the original readout of "uname -a" that would have been helpful information to you.
What I can say is though that the problematic setup had not been a fresh install, but an upgrade from xubuntu 8.04, with ubuntu-desktop added later.
Hope this helped even a little.

Hannes

$ uname -a
Linux desktop 2.6.27-7-generic #1 SMP Thu Oct 30 04:18:38 UTC 2008 i686 GNU/Linux

Hmm, so I was not able to reproduce this yet.

I was able to successfully copy a hierarchy of 160,640 files, 14,965 directories, comprising 5GB of data.

I do have 4GB of memory in this system (and 6GB of swap), so I'm highly unlikely to hit any out-of-memory errors.

Next, I'll try to reproduce this in a VM with some small, paltry amount of memory.

Out of curiosity, how much memory and swap do you have in your system, Michael Elkins?

 $ free

:-Dustin

$ free
             total used free shared buffers cached
Mem: 1033580 546700 486880 0 19608 226372
-/+ buffers/cache: 300720 732860
Swap: 3028212 0 3028212

The directory I am copying is only 9.3MB, 31 files, the largest is 4.6MB. The files it is failing on are all much less than 100KB.

I am having the same problem here. I have also noted that the files where the memory allocation error occurs are not being copied correctly (so it is not a benign message):

Computer: AMD Duron 1.6Ghz (32 bit only)

pjssilva@leia:~/Private$ uname -a
Linux leia 2.6.27-7-generic #1 SMP Tue Nov 4 19:33:20 UTC 2008 i686 GNU/Linux
pjssilva@leia:~/Private$ free
             total used free shared buffers cached
Mem: 1553240 1505180 48060 0 63996 1094684
-/+ buffers/cache: 346500 1206740
Swap: 4096532 4928 4091604

Can everyone who is experiencing this bug place a message with their CPU type? I am running an old AMD Duron (over-clocked to 1.9GHz), 32 bit only processor.

I have the feeling that the processor may be the problem. I have just tried the encrypted Private dir using Intrepid inside a virtual machine (virtual box 2.4). I get the same problem. It is always reproducible.

However, if I copy the virtual machine disk to another computer (a laptop) with an Intel processor (core 2 duo), the problem vanishes. Note that the main difference between the two copies of the virtual machine is the processor (as virtualbox delegates the instructions to the real processor).

Can the processor be the reason? Maybe encfs is using some optimized operations to speedup the cryptography?

Interesting proposition, Paulo. I am running an Intel Core2 Duo in 64
bit mode, and I have not been able to reproduce this on my hardware,
or in KVM's on this machine.

I will try it on an amd64 machine.

:-Dustin

I am using an older 32bit AMD processor, AMD XP 2500+ iirc.

> Error opening file '/home/me/Private/foo/benefits/guide.pdf': Cannot allocate memory

Please post any syslog content that gets written when you encounter this error.

I attach below the error messages I get when doing

pjssilva@leia:~$ cp -r /usr/lib/evolution Private/
cp: cannot create regular file `Private/evolution/2.24/conduits/libeaddress_conduit.so': Cannot allocate memory
cp: cannot create regular file `Private/evolution/2.24/libecontactlisteditor.so.0.0.0': Cannot allocate memory
cp: cannot create regular file `Private/evolution/2.24/plugins/liborg-gnome-bogo-junk-plugin.so': Cannot allocate memory
cp: cannot create regular file `Private/evolution/2.24/plugins/org-gnome-save-calendar.eplug': Cannot allocate memory
cp: cannot create regular file `Private/evolution/2.24/plugins/org-gnome-evolution-caldav.eplug': Cannot allocate memory
cp: cannot create regular file `Private/evolution/2.24/plugins/org-gnome-templates.eplug': Cannot allocate memory
cp: cannot create regular file `Private/evolution/2.24/plugins/liborg-gnome-prefer-plain.so': Cannot allocate memory
cp: cannot create regular file `Private/evolution/2.24/plugins/liborg-gnome-gw-account-setup.so': Cannot allocate memory
cp: cannot create regular file `Private/evolution/2.24/plugins/org-gnome-bogo-junk-plugin.eplug': Cannot allocate memory

Also, do you see this same problem if you use "cp -a" on the command
line, rather than through Nautilus?

:-Dustin

Same problem using cp or tar jxvf myfile.tar.bz2 on a machine running :
Linux box 2.6.27-7-generic #1 SMP Tue Nov 4 19:33:20 UTC 2008 i686 GNU/Linux
AMD Athlon XP2200.

But works on another machine, running same kernel,
Intel(R) Pentium(R) Dual CPU T2310 @ 1.46GHz

Dustin:

Yes, I tried all nautilus, "cp -r", "cp -a", and a hand made Python script that calls shutils.copyfile recursively in a tree. All fail.

On Tue, Nov 11, 2008 at 11:47 AM, Paulo J. S. Silva <email address hidden> wrote:
> Yes, I tried all nautilus, "cp -r", "cp -a", and a hand made Python
> script that calls shutils.copyfile recursively in a tree. All fail.

Hmm, this definitely sounds like a kernel issue. mhalcrow is
subscribed to this bug, who should be interested in this from the
kernel side. I'm still trying to reproduce it...

--
:-Dustin

Same problem here, using linux 2.6.27-7-generic on an AMD Athlon XP 1800+. Does anyone have this problem on non-AMD hardware?

Just to be more specific: Does anyone have this problem on non-"Athlon XP" hardware? In my case I have an old Duron, which is an Athlon XP with less cache.

I bet the problem is not present on Athlon 64 hardware (otherwise this bug would be flooded with reports). Maybe it is an error on some hardware optimized path to speed up ecryptfs? For example, Athlon XP does have SSE but it does not have SSE2 extensions.

Looks like it is the same sort of issue that we had here:

http://lkml.org/lkml/2008/7/28/207

It seems as if crypt_stat->key is not page-aligned on the affected archicture.
eCryptfs should not be assuming that part of the struct is page-aligned in the
first place. I will work on a patch to page_alloc() an intermediate buffer for
the scatterlist, just to keep things simple.

SRU Justification

Impact: ecryptfs can experience errors during file operations

Patch Description: Allocate 2 scatter/gather list elements when setting up the key transfer buffer.

Patch: http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-intrepid.git;a=commit;h=fc54e8c0062d75a7c44bc4edab59303bc410870d

Test Case: see bug description

It looks like this is not a bug in ecryptfs-utils, closing this task. Please reopen if that's wrong.

Accepted linux into intrepid-proposed, please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

It isn't clear to me which package from intrepid-proposed I am supposed to test.

The kernel from intrepid-proposed did not fix this problem.

I followed the steps in https://wiki.ubuntu.com/Testing/EnableProposed and ran
# apt-get install -t intrepid-proposed linux
# shutdown -r now new kernel

This is the version that was installed:
ii linux-image-generic 2.6.27.7.11 Generic Linux kernel image

me@desktop:~$ uname -a
Linux desktop 2.6.27-7-generic #1 SMP Tue Nov 4 19:33:20 UTC 2008 i686 GNU/Linux

me@desktop:~$ cp -r private/ Private/
cp: cannot create regular file `Private/private/archive/muttstuff/mutt-plugin/mutt_menu.h': Cannot allocate memory
...

Here is the output from dmesg:
[ 944.144646] write_tag_3_packet: Error generating scatterlist for crypt_stat session key; expected rc = 1; got rc = [-12]. key_rec->enc_key_size = [16]
[ 944.144661] ecryptfs_generate_key_packet_set: Error writing tag 3 packet
[ 944.144666] ecryptfs_write_headers_virt: Error generating key packet set; rc = [-12]
[ 944.144669] ecryptfs_write_metadata: Error whilst writing headers; rc = [-12]
[ 944.144675] Error writing headers; rc = [-12]

Michael-

It doesn't look like you're running the -proposed kernel, which is 2.6.27-10.20.

https://edge.launchpad.net/ubuntu/intrepid/+source/linux/2.6.27-10.20

:-Dustin

Michael-

And one more followup...

2.6.27-10.20 isn't built yet. It should be in place by Monday, Nov 24th. That's the one we'll need to verify, as it contains the proposed fix for this bug.

Cheers,
:-Dustin

Dustin, thanks for the confirmation. It looks like -10.20 has not propagated to all the mirrors yet. I will try again later.

Michael

Dustin, it seems that linux-2.6.27-10.20 still isn't on the Ubuntu server. What's going on? Any idea when it will be availyble?

Thanks, Martin

I have just got the upgrade to linux-image version 2.6.27.10.13. If fixed the bug.

Good work!

I'm marking the upstream ecryptfs userspace package "Invalid". The bug is in the kernel, and the fix has been built and verified.

Let's get this copied to Intrepid's updates ASAP.

Thanks,
:-Dustin

Please ignore my previous comment. It turned out that linux-image-2.6.27-10-generic (Version 2.6.27-10.20) is indeed in the intrepid-proposed repository, and it does fix this bug. It is a dependency of linux-image-generic (Version 2.6.27-10.13). I don't quite understand the reason behind the different numbering, but whatever. The bug is fixed, thanks a lot :-)

Martin