Ubuntu One Servers

Songs not downloading, even after a local and server rescan, but available via the web

Bug #571548 reported by Guillermo Gonzalez on 2010-04-29

136

This bug affects 24 people

	Status	Importance	Assigned to	Milestone
Ubuntu One Client	Status tracked in Trunk
Stable-1-2	Fix Released	Critical	Guillermo Gonzalez	Ubuntu One Client 1.2.2
Trunk	Fix Released	Critical	Guillermo Gonzalez
Ubuntu One Servers	Invalid	Critical	Guillermo Gonzalez
ubuntuone-client (Ubuntu)	Fix Released	Critical	dobey	Ubuntu maverick-alpha-2
Lucid	Fix Released	Undecided	Unassigned
Maverick	Fix Released	Critical	dobey	Ubuntu maverick-alpha-2

Bug Description

jono and bac reported a very similar issue, songs (or entire albums) not downloading but available via the web.
So, the files are on the server but the client isn't downloading them.

-----------------------------------------------

WORK AROUND:
Open Applicaitons->Accessories->Terminal and run: touch ~/.ubuntuone/Purchased\ from\ Ubuntu\ One/placeholder; u1sdtool -q; u1sdtool -c

This will put a "placeholder" file in your ~/.ubuntuone/Purchased from Ubuntu One/ folder and then force the syncdaemon to reconnect and try to sync that folder.

-----------------------------------------------

TEST CASE:
Need to clients to test. Client A & B which are referenced below.

1) Sync Client A
# then start a single client (a.k.a: A) and create a UDF, e.g:
u1sdtool -c
u1sdtool -s (wait to reach idle)
u1sdtool --create-folder=$HOME/MyUDF
u1sdtool -s (wait to reach idle)
# create some files and dirs inside $HOME/MyUDF
mkdir $HOME/MyUDF/subdir
mkdir $HOME/MyUDF/subdir2
touch $HOME/MyUDF/subdir/bar
touch $HOME/MyUDF/subdir2/foo
# create a file in the root
touch $HOME/MyUDF/foo
u1sdtool -s (wait to reach idle)

2) Sync Client B
# start a second client (probably a different machine), a.k.a: B
u1sdtool -c
u1sdtool -s (wait to reach idle)
# it should download all the files/dirs created in client A
# quit the client B
u1sdtool -q

3) Sync new file on Client A
# on client A, create a new file/dir in the root
touch $HOME/MyUDF/bar
u1sdtool -s (wait to reach idle)
# quit and restart the client A to make sure everything it's ok
u1sdtool -q
u1sdtool -c
u1sdtool -s (wait to reach idle)
# now get the info of the UDF root metadata
u1sdtool --info=$HOME/MyUDF
# keep the output around, we are going to need the 'server_hash' value for the next step

4) Run md_server.py script (http://launchpadlibrarian.net/48350344/md_server.py)
# on client B, we need to fake a interrupted download
# use the server_hash from the previous step (from client A)
python md_server.py $HOME/MyUDF <server_hash>

5) Sync Client B (again)
# still on client B, let's start syncdaemon again
u1sdtool -c
u1sdtool -s (wait to reach idle)
# with the current version of ubuntuone-client we should see that the new file ($HOME/MyUDF/bar) wasn't downloaded on client B

6) Sync Client B with fix
# using the fix on #571548, we should get the new file downloaded to client B
# to check this, run:
u1sdtool -q
# get a fresh copy of trunk or the branch linked to #571548 and run:
ln -s /usr/lib/python2.6/dist-packages/ubuntuone/storageprotocol ubuntuone/storageprotocol
PYTHONPATH=. bin/ubuntuone-syncdaemon &
u1sdtool -c
u1sdtool -s (wait to reach idle)

Expected result: $HOME/MyUDF/bar is on Client B

-----------------------------------------------

See original description

Tags:

Related branches

lp:~verterok/ubuntuone-client/fix-571548

Merged into lp:ubuntuone-client/stable-1-2 at revision 511

Rodrigo Moya (community): Approve on 2010-05-06

Tim Cole (community): Approve on 2010-05-04

Natalia Bidart (community): Approve on 2010-04-30

lp:~verterok/ubuntuone-client/fix-571548-trunk

Merged into lp:ubuntuone-client at revision 515

Tim Cole (community): Approve on 2010-05-04

Natalia Bidart (community): Approve on 2010-04-30

lp:ubuntu/lucid-updates/ubuntuone-client

Guillermo Gonzalez (verterok) on 2010-04-29

tags:	added: foundations+ removed: founda
Changed in ubuntuone-servers:
assignee:	nobody → Ubuntu One Foundations+ team (ubuntuone-foundations+)

Revision history for this message

Guillermo Gonzalez (verterok) wrote on 2010-04-29:

To work around this, the artist folder was created manually, and a file was created inside it.
IMO, only the first step was needed to trigger the download.

Revision history for this message

Guillermo Gonzalez (verterok) wrote on 2010-04-29:

Dump of syncdaemon metadata before creating the artist directory.

Revision history for this message

Guillermo Gonzalez (verterok) wrote on 2010-04-29:

syncdaemon metadata dump after downloading the missing files

Guillermo Gonzalez (verterok) on 2010-04-29

Changed in ubuntuone-servers:
status:	New → Invalid
Changed in ubuntuone-client:
importance:	Undecided → Critical
assignee:	nobody → Ubuntu One Foundations+ team (ubuntuone-foundations+)
status:	New → Triaged
assignee:	Ubuntu One Foundations+ team (ubuntuone-foundations+) → Guillermo Gonzalez (verterok)

Revision history for this message

Facundo Batista (facundo) wrote on 2010-04-29:

We need to handle Roots (for U1, Shares and UDFs) in a special way when they're in SERVER state after an interrumpted syncdaemon when it was downloading new data.

That should be done in LR, and tests for this case for the three roots should be added.

This will be done by Guillermo, we talked extensively about this this morning.

Changed in ubuntuone-servers:
assignee:	Ubuntu One Foundations+ team (ubuntuone-foundations+) → Guillermo Gonzalez (verterok)

Guillermo Gonzalez (verterok) on 2010-05-01

Changed in ubuntuone-client:
status:	Triaged → In Progress

Guillermo Gonzalez (verterok) on 2010-05-04

tags:

added: u1-lucid-sru

Guillermo Gonzalez (verterok) on 2010-05-05

Changed in ubuntuone-client:
status:	In Progress → Fix Committed
Changed in ubuntuone-client (Ubuntu):
importance:	Undecided → Critical
status:	New → Confirmed
milestone:	none → lucid-updates
assignee:	nobody → Rodney Dawes (dobey)

Joshua Hoover (joshuahoover) on 2010-05-11

description:

updated

Joshua Hoover (joshuahoover) on 2010-05-11

description:

updated

Revision history for this message

Joshua Hoover (joshuahoover) wrote on 2010-05-12:

md_server.py Edit (1.2 KiB, text/x-python)

md_server.py script used in test case.

description:	updated
description:	updated

Joshua Hoover (joshuahoover) on 2010-05-12

description:

updated

Revision history for this message

Joshua Hoover (joshuahoover) wrote on 2010-05-12:

Ran test case and got expected results with the branch. I'll test the proposed update once it's there.

description:

updated

Guillermo Gonzalez (verterok) on 2010-05-13

visibility:

private → public

Revision history for this message

Otto Kekäläinen (otto) wrote on 2010-05-13:

I bought the album Crazy Love by Michael Bublé around the first of May 2010. This was my first and possible last purchase from U1 music store.

The payment went ok and the files appeared in the cloud (visible from the U1 web interface) but never synced to any of my local computers. The folder "~/.ubuntuone/Purchased from Ubuntu One/" stayed empty no matter what I tried (e.g. reinstalling U1 and my account). Other files in the normal Ubuntu One folder synced fine. My computer is on several hours every day so it wasn't a matter of waiting long enough. Command u1sdtool -s showed idle all the time.

Finally today I got help from verterok and alecu on the #ubuntuone IRC channel. Running mkdir "Michael Bublé" in the folder "~/.ubuntuone/Purchased from Ubuntu One/" first destroyed the files in the cloud. Luckily there was an option to download the files again from the store, so clicking "download it again" in the U1 music store interface in Rhythmbox fixed it (except for one song, see below). Now the songs are in the cloud and they synced to one of my local computers.

I went to check the situation on another computer where I have the same U1 account: the folder "~/.ubuntuone/Purchased from Ubuntu One/" on that computer was still empty but running "touch test" in that folder somehow poked the U1 daemon and the sync started.

There is just one problem left: when I click to download the song "Baby [You've Got What It Takes] [with Sharon Jones & Dap-Kings]" (by Michael Bublé on album Crazy Love) in the Rhythmbox U1 music store interface, the page just reloads but no transfer is started.. I have to complain about that in another bug report.

But I guess the workaround to a local folder not syncing is to run "touch newfile" in it to wake up the sync calculation.

Joshua Hoover (joshuahoover) on 2010-05-13

description:

updated

Roman Yepishev (rye) on 2010-05-18

tags:

added: current-trouble

Revision history for this message

dobey (dobey) wrote on 2010-06-01:

Patch in the 1.2.1-0ubuntu3 upload Edit (9.3 KiB, text/plain)

Revision history for this message

John Dong (jdong) wrote on 2010-06-02:

#10

ACK from ubuntu-sru

Revision history for this message

Martin Pitt (pitti) wrote on 2010-06-02: Please test proposed package

#11

Accepted ubuntuone-client into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in ubuntuone-client (Ubuntu Lucid):
status:	New → Fix Committed
tags:	added: verification-needed

Revision history for this message

Martin Pitt (pitti) wrote on 2010-06-02:

#12

Incidentally I purchased an album this morning (mp3), and as usual it got stuck in the "Queueing" state for a bit (some 30 minutes?) and then switched to "Transferring to Ubuntu One..." without further action (now 3 hours ago). I updated to this proposed version and restarted session (and also restarted ubuntuone in the prefs dialog just for testing), but it still doesn't start downloading. I see the files on one.u.c., but not in ~/.ubuntuone/Purchased\ from\ Ubuntu\ One/. Wasn't that what this update was supposed to fix?

Revision history for this message

Geoff123 (gsking1) wrote on 2010-06-02:

#13

I just purchase an album today and have run into this problem. I've tried the proposed update and it is still not working. I also tried the workaround at the top of this page, rebooting, restarting the ubuntuone client, etc. but those didn't help. About the same behavior as Martin Pitt above. I can download the songs from the ubuntu files webpage, but there appears to be no other syncing going on.

Revision history for this message

Matt Griffin (mattgriffin) wrote on 2010-06-02:

#14

Geoff123: File sync between the Ubuntu One cloud and your desktop is currently very slow. This is noted on our Status page: https://wiki.ubuntu.com/UbuntuOne/Status#Files

In addition to purchased music, this affects all files that customers are trying to sync. The issue lies on our servers and we're close to a solution. Workarounds on your desktop for this problem will probably not work.

In the meantime, purchased songs should be available for download from your personal cloud at https://one.ubuntu.com/files immediately after purchase. It's not ideal, but it will allow you to get your purchased music until we fix the slow file sync issue.

Revision history for this message

Bertrand Frottier (bertrand-frottier) wrote on 2010-06-04:

#15

Just purchased an album. Luckily for me the "placeholder workaround" did work fine.

Revision history for this message

Daniel Holbach (dholbach) wrote on 2010-06-05:

#16

With the new ubuntuone-client packages, the workaround seems to work for me.

Revision history for this message

Paul D Smyth (pauldsmyth) wrote on 2010-06-05:

#17

Applied the new client, seemed to catch everything up. Purchased an album, downloaded immediately. Seems to be working properly now

papukaija (papukaija) on 2010-06-07

tags:

added: patch

Martin Pitt (pitti) on 2010-06-08

tags:

added: verification-done
removed: verification-needed

Revision history for this message

Martin Pitt (pitti) wrote on 2010-06-11:

#18

Please fix this in maverick ASAP (or close the maverick task if it already is).

Changed in ubuntuone-client (Ubuntu):
milestone:	lucid-updates → maverick-alpha-2
milestone:	maverick-alpha-2 → none
status:	Confirmed → In Progress

Revision history for this message

Launchpad Janitor (janitor) wrote on 2010-06-11:

#19

This bug was fixed in the package ubuntuone-client - 1.2.1-0ubuntu3

---------------
ubuntuone-client (1.2.1-0ubuntu3) lucid-proposed; urgency=low

* Add fix-571548.patch
- Fixes songs not downloading but available on web issues (LP: #571548)
-- Rodney Dawes <email address hidden> Tue, 01 Jun 2010 15:31:58 -0400

Changed in ubuntuone-client (Ubuntu Lucid):
status:	Fix Committed → Fix Released

Martin Pitt (pitti) on 2010-06-11

Changed in ubuntuone-client (Ubuntu Maverick):
milestone:	none → maverick-alpha-2

Revision history for this message

dobey (dobey) wrote on 2010-06-11:

#20

The fix is built in maverick already, but the packages seem to still be pending publication to the archive. Not sure why.

Changed in ubuntuone-client (Ubuntu Maverick):
status:	In Progress → Fix Released

Revision history for this message

yajo (yajodj) wrote on 2010-09-17:

#21

Same bug here.
Running Ubuntu 10.04.
Workaround worked for me too. Thanks.

Revision history for this message

E King (eakst7) wrote on 2010-09-23:

#22

Sorry for the "me too", but this is still broken for me on 10.04. This was my first attempt at using the music store. The "placeholder" workaround gave it the kick it needed.

Changed in ubuntuone-client (Ubuntu Lucid):
status:	Fix Released → Confirmed

Revision history for this message

Justin (ja-space) wrote on 2010-09-23:

#23

Just got same error with 10.04 Lucid. First go at buying music so really disappointing, but work around did work...

Revision history for this message

dobey (dobey) wrote on 2010-09-23:

#24

If you think you are seeing this same issue with lucid, with ALL updates applied please file a new bug. This bug has been fixed and verified, so if you're seeing a similar issue, we should track it in a new bug, which might be a regression. Please file a new bug and ensure appropriate logs are attached to it. Please don't reopen bugs that were fixed months ago. Thanks.