VMware Cloud Community
Ghada1
Contributor
Contributor

How to upload a file in Swift Storage system using python?

I want to upload some files in Swift, having each around 1 giga bytes as sizes. I executed this code in python (for one file to upload).


I got this error : Object PUT failed 503 Service Unavailable swift. What can be the reason of the error ?

Thank you.

Tags (3)
Reply
0 Kudos
8 Replies
xiangfeiz
VMware Employee
VMware Employee

Hi there, are you using VIO? I'm asking because VIO does not enable Swift by default. If you have enabled Swift, can you try using python swift client to get more detail log? For an example, you can run the following command:

swift --debug upload <container name> <file name>

Please post the console output here.

Thank you.

Reply
0 Kudos
Ghada1
Contributor
Contributor

Hi xiangfeiz,

I am not using VIO. To install Swift, I followed the documentation : SAIO - Swift All In One — swift 2.7.1.dev158 documentation

What are the benefits of using VIO ?

THis is the console output after running the command :

output.PNG

Reply
0 Kudos
Ghada1
Contributor
Contributor

  In the previous image, I forgot to mention the credentials.

Please, consider this output ;

INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): 127.0.0.1

DEBUG:requests.packages.urllib3.connectionpool:"GET /auth/v1.0 HTTP/1.1" 200 0

DEBUG:swiftclient:REQ: curl -i http://127.0.0.1:8080/auth/v1.0 -X GET

DEBUG:swiftclient:RESP STATUS: 200 OK

DEBUG:swiftclient:RESP HEADERS: {u'Content-Length': u'0', u'X-Trans-Id': u'tx4770577739904286aa2a0-005748bacb', u'X-Auth-Token-Expires': u'76979', u'X-Auth-Token': u'AUTH_tk20369f86368242079f65a6229c0188e8', u'X-Storage-Token': u'AUTH_tk20369f86368242079f65a6229c0188e8', u'Date': u'Fri, 27 May 2016 21:23:23 GMT', u'X-Storage-Url': u'http://127.0.0.1:8080/v1/AUTH_test', u'Content-Type': u'text/html; charset=UTF-8'}

INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): 127.0.0.1

DEBUG:requests.packages.urllib3.connectionpool:"PUT /v1/AUTH_test/container HTTP/1.1" 202 76

DEBUG:swiftclient:REQ: curl -i http://127.0.0.1:8080/v1/AUTH_test/container -X PUT -H "Content-Length: 0" -H "X-Auth-Token: AUTH_tk20369f86368242079f65a6229c0188e8"

DEBUG:swiftclient:RESP STATUS: 202 Accepted

DEBUG:swiftclient:RESP HEADERS: {u'Date': u'Fri, 27 May 2016 21:23:23 GMT', u'Content-Length': u'76', u'Content-Type': u'text/html; charset=UTF-8', u'X-Trans-Id': u'txf6d5541faffa4cc5a7ff2-005748bacb'}

DEBUG:swiftclient:RESP BODY: <html><h1>Accepted</h1><p>The request is accepted for processing.</p></html>

INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): 127.0.0.1

DEBUG:requests.packages.urllib3.connectionpool:"GET /auth/v1.0 HTTP/1.1" 200 0

DEBUG:swiftclient:REQ: curl -i http://127.0.0.1:8080/auth/v1.0 -X GET

DEBUG:swiftclient:RESP STATUS: 200 OK

DEBUG:swiftclient:RESP HEADERS: {u'Content-Length': u'0', u'X-Trans-Id': u'txf5b6fa7737ee4ffab2503-005748bacb', u'X-Auth-Token-Expires': u'76979', u'X-Auth-Token': u'AUTH_tk20369f86368242079f65a6229c0188e8', u'X-Storage-Token': u'AUTH_tk20369f86368242079f65a6229c0188e8', u'Date': u'Fri, 27 May 2016 21:23:23 GMT', u'X-Storage-Url': u'http://127.0.0.1:8080/v1/AUTH_test', u'Content-Type': u'text/html; charset=UTF-8'}

INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): 127.0.0.1

DEBUG:requests.packages.urllib3.connectionpool:"HEAD /v1/AUTH_test/container/ubuntu.iso HTTP/1.1" 404 0

INFO:swiftclient:REQ: curl -i http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso -I -H "X-Auth-Token: AUTH_tk20369f86368242079f65a6229c0188e8"

INFO:swiftclient:RESP STATUS: 404 Not Found

INFO:swiftclient:RESP HEADERS: {u'Date': u'Fri, 27 May 2016 21:23:23 GMT', u'Content-Length': u'0', u'Content-Type': u'text/html; charset=UTF-8', u'X-Trans-Id': u'tx07cfd6d43f8b437980705-005748bacb'}

ERROR:swiftclient:Object HEAD failed: http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso 404 Not Found

Traceback (most recent call last):

  File "/home/ghada/python-swiftclient/swiftclient/client.py", line 1530, in _retry

    service_token=self.service_token, **kwargs)

  File "/home/ghada/python-swiftclient/swiftclient/client.py", line 1144, in head_object

    raise ClientException.from_response(resp, 'Object HEAD failed', body)

ClientException: Object HEAD failed: http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso 404 Not Found

DEBUG:requests.packages.urllib3.connectionpool:"PUT /v1/AUTH_test/container/ubuntu.iso HTTP/1.1" 503 118

INFO:swiftclient:REQ: curl -i http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso -X PUT -H "x-object-meta-mtime: 1464129494.233180" -H "Content-Length: 1485881344" -H "Content-Type: " -H "X-Auth-Token: AUTH_tk20369f86368242079f65a6229c0188e8"

INFO:swiftclient:RESP STATUS: 503 Service Unavailable

INFO:swiftclient:RESP HEADERS: {u'Date': u'Fri, 27 May 2016 21:23:23 GMT', u'Content-Length': u'118', u'Content-Type': u'text/html; charset=UTF-8', u'X-Trans-Id': u'txf3c53fa2ce33483695004-005748bacb'}

INFO:swiftclient:RESP BODY: <html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>

DEBUG:requests.packages.urllib3.connectionpool:"PUT /v1/AUTH_test/container/ubuntu.iso HTTP/1.1" 503 118

INFO:swiftclient:REQ: curl -i http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso -X PUT -H "x-object-meta-mtime: 1464129494.233180" -H "Content-Length: 1485881344" -H "Content-Type: " -H "X-Auth-Token: AUTH_tk20369f86368242079f65a6229c0188e8"

INFO:swiftclient:RESP STATUS: 503 Service Unavailable

INFO:swiftclient:RESP HEADERS: {u'Date': u'Fri, 27 May 2016 21:23:34 GMT', u'Content-Length': u'118', u'Content-Type': u'text/html; charset=UTF-8', u'X-Trans-Id': u'tx1eca4cb95f5f4f9f9b9a2-005748bad6'}

INFO:swiftclient:RESP BODY: <html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>

DEBUG:requests.packages.urllib3.connectionpool:"PUT /v1/AUTH_test/container/ubuntu.iso HTTP/1.1" 503 118

INFO:swiftclient:REQ: curl -i http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso -X PUT -H "x-object-meta-mtime: 1464129494.233180" -H "Content-Length: 1485881344" -H "Content-Type: " -H "X-Auth-Token: AUTH_tk20369f86368242079f65a6229c0188e8"

INFO:swiftclient:RESP STATUS: 503 Service Unavailable

INFO:swiftclient:RESP HEADERS: {u'Date': u'Fri, 27 May 2016 21:23:45 GMT', u'Content-Length': u'118', u'Content-Type': u'text/html; charset=UTF-8', u'X-Trans-Id': u'txfb9307ddff534557b1695-005748bae1'}

INFO:swiftclient:RESP BODY: <html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>

DEBUG:requests.packages.urllib3.connectionpool:"PUT /v1/AUTH_test/container/ubuntu.iso HTTP/1.1" 503 118

INFO:swiftclient:REQ: curl -i http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso -X PUT -H "x-object-meta-mtime: 1464129494.233180" -H "Content-Length: 1485881344" -H "Content-Type: " -H "X-Auth-Token: AUTH_tk20369f86368242079f65a6229c0188e8"

INFO:swiftclient:RESP STATUS: 503 Service Unavailable

INFO:swiftclient:RESP HEADERS: {u'Date': u'Fri, 27 May 2016 21:23:59 GMT', u'Content-Length': u'118', u'Content-Type': u'text/html; charset=UTF-8', u'X-Trans-Id': u'tx00427d22a5e1410ea30ba-005748baef'}

INFO:swiftclient:RESP BODY: <html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>

DEBUG:requests.packages.urllib3.connectionpool:"PUT /v1/AUTH_test/container/ubuntu.iso HTTP/1.1" 503 118

INFO:swiftclient:REQ: curl -i http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso -X PUT -H "x-object-meta-mtime: 1464129494.233180" -H "Content-Length: 1485881344" -H "Content-Type: " -H "X-Auth-Token: AUTH_tk20369f86368242079f65a6229c0188e8"

INFO:swiftclient:RESP STATUS: 503 Service Unavailable

INFO:swiftclient:RESP HEADERS: {u'Date': u'Fri, 27 May 2016 21:24:16 GMT', u'Content-Length': u'118', u'Content-Type': u'text/html; charset=UTF-8', u'X-Trans-Id': u'tx53e736736510459b923cf-005748bb00'}

INFO:swiftclient:RESP BODY: <html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>

DEBUG:requests.packages.urllib3.connectionpool:"PUT /v1/AUTH_test/container/ubuntu.iso HTTP/1.1" 503 118

INFO:swiftclient:REQ: curl -i http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso -X PUT -H "x-object-meta-mtime: 1464129494.233180" -H "Content-Length: 1485881344" -H "Content-Type: " -H "X-Auth-Token: AUTH_tk20369f86368242079f65a6229c0188e8"

INFO:swiftclient:RESP STATUS: 503 Service Unavailable

INFO:swiftclient:RESP HEADERS: {u'Date': u'Fri, 27 May 2016 21:24:42 GMT', u'Content-Length': u'118', u'Content-Type': u'text/html; charset=UTF-8', u'X-Trans-Id': u'txc087c0fab26342cba2fa0-005748bb1a'}

INFO:swiftclient:RESP BODY: <html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>

ERROR:swiftclient:Object PUT failed: http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso 503 Service Unavailable  [first 60 chars of response] <html><h1>Service Unavailable</h1><p>The server is currently

Traceback (most recent call last):

  File "/home/ghada/python-swiftclient/swiftclient/client.py", line 1530, in _retry

    service_token=self.service_token, **kwargs)

  File "/home/ghada/python-swiftclient/swiftclient/client.py", line 1256, in put_object

    raise ClientException.from_response(resp, 'Object PUT failed', body)

ClientException: Object PUT failed: http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso 503 Service Unavailable  [first 60 chars of response] <html><h1>Service Unavailable</h1><p>The server is currently

ERROR:swiftclient.service:Object PUT failed: http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso 503 Service Unavailable  [first 60 chars of response] <html><h1>Service Unavailable</h1><p>The server is currently

Traceback (most recent call last):

  File "/home/ghada/python-swiftclient/swiftclient/service.py", line 1975, in _upload_object_job

    response_dict=obr

  File "/home/ghada/python-swiftclient/swiftclient/client.py", line 1674, in put_object

    response_dict=response_dict)

  File "/home/ghada/python-swiftclient/swiftclient/client.py", line 1530, in _retry

    service_token=self.service_token, **kwargs)

  File "/home/ghada/python-swiftclient/swiftclient/client.py", line 1256, in put_object

    raise ClientException.from_response(resp, 'Object PUT failed', body)

ClientException: Object PUT failed: http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso 503 Service Unavailable  [first 60 chars of response] <html><h1>Service Unavailable</h1><p>The server is currently

Object PUT failed: http://127.0.0.1:8080/v1/AUTH_test/container/ubuntu.iso 503 Service Unavailable  [first 60 chars of response] <html><h1>Service Unavailable</h1><p>The server is currently

Reply
0 Kudos
admin
Immortal
Immortal

Hi Ghada1,

VIO includes standard defcore certified openstack along with the appropriate drivers (nova-vcenter driver, cinder-vmdk driver, neutron-nsx or neutron-dvs driver, glance-vmdk driver...) necessary to run openstack on VMware infrastructure. All of this is opensource, so what's the benefit of VIO? Some of the benefits are,

  1. Deploying openstack with VIO is extremely simple and quick.
  2. Out of the box architecture: VIO is deployment in a production ready, fully redundant HA architecture with all the necessary support services (rabbit mq, memcache, clustered galera db). One does not have to worry about how to architect the different services.
  3. Upgrading openstack across major releases (say for example Kilo to Mitaka) is seamless and automated, with ability to roll back.
  4. VIO includes simple workflows for day 2 operations such as patching, adding/removing compute, storage.
  5. Fully supported by VMware - unified support for both the openstack and the vmware components.
  6. VIO comes with recommendations of exact version of underlying components with which it has been extensively tested and verified. For example VIO 2.0.1 <-> vSphere 6.0 <-> NSX 6.2.2 etc.
  7. Leverage existing expertise - if you know how to operate vsphere you don't have to learn anything new to deploy and operate openstack. VIO provides a plugin for vsphere web client which allows you to use familiar constructs such as clusters, datastore, distributed switches etc to deploy and operate openstack.

Please also checkout the VIO youtube channel for some cool demo videos of the product: VMware Integrated OpenStack - YouTube

I'll also send a direct message to you with my email so that you can connect with me directly if you have more questions.

Reply
0 Kudos
xiangfeiz
VMware Employee
VMware Employee

It is not clear why 503 service unavailable was returned when looking at the log. You can run "swift stat" and "swift capabilities" to see if there is anything unusual. And please check /var/log/syslog right after you run the swift command to see if there is anything wrong.

Reply
0 Kudos
Ghada1
Contributor
Contributor

Thank you xiangfeiz

Actually, I allocated to Swift only 1 GB of storage capacity while the installation by using this command : sudo truncate -s 1GB /srv/swift-disk.

So, the storage space is almost full. How can I increase the space allocated to Swift ??

Reply
0 Kudos
xiangfeiz
VMware Employee
VMware Employee

Please backup your data first if there is anything import.

Assuming you use the SAIO document, you can increase the swift partition size using truncate and xfs_growfs

sudo truncate -s 10GB /srv/swift-disk

xfs_growfs /mnt/sdb1

You can check the new size of the partition using "df -h".

Reply
0 Kudos
Xiaoliangl
Contributor
Contributor

could you provide your swift server logs? normally the logs should be inside /opt/stack/logs, and swift server component log file names begin with "s-".

Reply
0 Kudos