Another variant of this? This time, I'm not seeing that an image was partially uploaded, but it looks like glance restarted during an API call from gs3 that led to an exception without a retry
[From glance-simplestreams-sync-0/var/log/glance-simplestreams-sync.log]
ClientException: Object GET failed: http://10.245.201.183:80:80/swift/v1/simplestreams/data/streams/v1/auto.sync.json 404 Not Found NoSuchKey
DEBUG * 05-19 16:37:52 [PID:16871] * glanceclient.common.http * curl -i -X GET -H 'X-Auth-Token: 51228ff97043447b94f3f2db09e0c45f' -H 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' http://10.245.201.207:9292/v1/images/detail?limit=20
ERROR * 05-19 16:37:53 [PID:16871] * root * Exception during do_sync
Traceback (most recent call last):
File "/usr/share/glance-simplestreams-sync/glance-simplestreams-sync.py", line 444, in main
do_sync(charm_conf, status_exchange)
File "/usr/share/glance-simplestreams-sync/glance-simplestreams-sync.py", line 215, in do_sync
tmirror.sync(smirror, path=initial_path)
File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py", line 91, in sync
return self.sync_index(reader, path, data, content)
File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py", line 254, in sync_index
self.sync(reader, path=epath)
File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py", line 89, in sync
return self.sync_products(reader, path, data, content)
File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py", line 269, in sync_products
target = self.load_products(path, content_id)
File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/glance.py", line 172, in load_products
for image in images:
File "/usr/lib/python2.7/dist-packages/glanceclient/v1/images.py", line 174, in paginate
images = self._list(url, "images")
File "/usr/lib/python2.7/dist-packages/glanceclient/common/base.py", line 53, in _list
resp, body = self.api.json_request('GET', url)
File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 266, in json_request
resp, body_iter = self._http_request(url, method, **kwargs)
File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 226, in _http_request
resp = conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1051, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 415, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 379, in _read_status
raise BadStatusLine(line)
BadStatusLine: ''
And on glance:
[from glance-0/var/log/juju/unit-glance-0.log]
2016-05-19 16:37:54 INFO ceph-relation-changed glance-api stop/waiting
2016-05-19 16:37:54 INFO ceph-relation-changed glance-api start/running, process 27074
2016-05-19 16:37:54 INFO ceph-relation-changed glance-api stop/waiting
2016-05-19 16:37:54 INFO ceph-relation-changed glance-api start/running, process 27093
2016-05-19 16:37:54 INFO ceph-relation-changed glance-registry stop/waiting
2016-05-19 16:37:55 INFO ceph-relation-changed glance-registry start/running, process 27113
Another variant of this? This time, I'm not seeing that an image was partially uploaded, but it looks like glance restarted during an API call from gs3 that led to an exception without a retry
[From glance- simplestreams- sync-0/ var/log/ glance- simplestreams- sync.log] 10.245. 201.183: 80:80/swift/ v1/simplestream s/data/ streams/ v1/auto. sync.json 404 Not Found NoSuchKey common. http * curl -i -X GET -H 'X-Auth-Token: 51228ff97043447 b94f3f2db09e0c4 5f' -H 'Content-Type: application/json' -H 'User-Agent: python- glanceclient' http:// 10.245. 201.207: 9292/v1/ images/ detail? limit=20 glance- simplestreams- sync/glance- simplestreams- sync.py" , line 444, in main sync(charm_ conf, status_exchange) glance- simplestreams- sync/glance- simplestreams- sync.py" , line 215, in do_sync sync(smirror, path=initial_path) python2. 7/dist- packages/ simplestreams/ mirrors/ __init_ _.py", line 91, in sync index(reader, path, data, content) python2. 7/dist- packages/ simplestreams/ mirrors/ __init_ _.py", line 254, in sync_index sync(reader, path=epath) python2. 7/dist- packages/ simplestreams/ mirrors/ __init_ _.py", line 89, in sync products( reader, path, data, content) python2. 7/dist- packages/ simplestreams/ mirrors/ __init_ _.py", line 269, in sync_products products( path, content_id) python2. 7/dist- packages/ simplestreams/ mirrors/ glance. py", line 172, in load_products python2. 7/dist- packages/ glanceclient/ v1/images. py", line 174, in paginate python2. 7/dist- packages/ glanceclient/ common/ base.py" , line 53, in _list json_request( 'GET', url) python2. 7/dist- packages/ glanceclient/ common/ http.py" , line 266, in json_request request( url, method, **kwargs) python2. 7/dist- packages/ glanceclient/ common/ http.py" , line 226, in _http_request python2. 7/httplib. py", line 1051, in getresponse begin() python2. 7/httplib. py", line 415, in begin python2. 7/httplib. py", line 379, in _read_status
ClientException: Object GET failed: http://
DEBUG * 05-19 16:37:52 [PID:16871] * glanceclient.
ERROR * 05-19 16:37:53 [PID:16871] * root * Exception during do_sync
Traceback (most recent call last):
File "/usr/share/
do_
File "/usr/share/
tmirror.
File "/usr/lib/
return self.sync_
File "/usr/lib/
self.
File "/usr/lib/
return self.sync_
File "/usr/lib/
target = self.load_
File "/usr/lib/
for image in images:
File "/usr/lib/
images = self._list(url, "images")
File "/usr/lib/
resp, body = self.api.
File "/usr/lib/
resp, body_iter = self._http_
File "/usr/lib/
resp = conn.getresponse()
File "/usr/lib/
response.
File "/usr/lib/
version, status, reason = self._read_status()
File "/usr/lib/
raise BadStatusLine(line)
BadStatusLine: ''
And on glance: 0/var/log/ juju/unit- glance- 0.log] changed glance-api stop/waiting changed glance-api start/running, process 27074 changed glance-api stop/waiting changed glance-api start/running, process 27093 changed glance-registry stop/waiting changed glance-registry start/running, process 27113
[from glance-
2016-05-19 16:37:54 INFO ceph-relation-
2016-05-19 16:37:54 INFO ceph-relation-
2016-05-19 16:37:54 INFO ceph-relation-
2016-05-19 16:37:54 INFO ceph-relation-
2016-05-19 16:37:54 INFO ceph-relation-
2016-05-19 16:37:55 INFO ceph-relation-