A wrapper for OpenStack Object Storage v1 (aka Swift). Swift provides redundant storage similar to AWS S3. This gem is a wrapper around the Object # Storage RESTful API and provides the ability to manage containers (directories) and objects (files).
Example:
require 'pp' require 'att/swift' auth_uri = 'http://swift.example/auth/' swift = ATT::Swift.new auth_uri, 'username', 'password' pp swift.containers
See also docs.openstack.org/api/openstack-object-storage/1.0/content/
Creates a new Swift object for communication with
the server at auth_uri. user and
key are your credentials.
For AT&T Cloud storage:
swift = ATT::Swift.new \ 'https://data.iad1.attstorage.com/auth/', 'tennant:username', 'password'
# File lib/att/swift.rb, line 67 def initialize auth_uri, user, key auth_uri = URI auth_uri unless URI === auth_uri @auth_uri = auth_uri @user = user @key = key @http = Net::HTTP::Persistent.new 'att-swift' @storage_uri = nil @auth_token = nil end
Authenticates you with the swift server. This method is called automatically by other API methods.
# File lib/att/swift.rb, line 84 def authenticate return if @auth_token res = get @auth_uri + 'v1.0', 'X-Auth-User' => @user, 'X-Auth-Key' => @key case res when Net::HTTPSuccess @auth_token = res['X-Auth-Token'] storage_uri = res['X-Storage-Url'] storage_uri << '/' unless storage_uri.end_with? '/' @storage_uri = URI storage_uri @http.override_headers['X-Auth-Token'] = @auth_token else raise Error.new(res, 'authentication failed') end end
Like #paginate_objects,
but yields chunks of up to limit object information at a time
from the swift server to the given block. marker may be used
to start pagination after a particular entry.
swift.chunk_objects 'container' do |object_infos| object_infos.each do |object_info| # ... end end
# File lib/att/swift.rb, line 114 def chunk_objects container, marker = nil, limit = 1_000 return enum_for __method__, container, marker, limit unless block_given? loop do chunk = objects container, marker, limit break if chunk.empty? yield chunk marker = chunk.last['name'] end self end
Retrieves the containers for your server. Example output:
[{"name" => "public_bucket", "count" => 2, "bytes" => 9},
{"name" => "public_bucket_segments", "count" => 22, "bytes" => 21875}]
# File lib/att/swift.rb, line 136 def containers marker = nil, limit = nil authenticate params = { 'format' => 'json' } params['marker'] = marker if marker params['limit'] = limit if limit uri = @storage_uri + "?#{escape_params params}" res = get uri case res when Net::HTTPSuccess then JSON.parse res.body else raise Error.new(res, 'error listing containers') end end
Copies source_object in source_container to
destination_object in destination_container.
Returns true if the copy was successful, false if the source object was not found or the destination container did not exist.
# File lib/att/swift.rb, line 162 def copy_object source_container, source_object, destination_container, destination_object authenticate source_path = "#{source_container}/#{source_object}" destination_path = "#{destination_container}/#{destination_object}" uri = @storage_uri + destination_path req = Net::HTTP::Put.new uri.request_uri req['X-Copy-From'] = "/#{source_path}" req.body = '' res = @http.request uri, req case res when Net::HTTPSuccess then true when Net::HTTPNotFound false else raise Error.new(res, "error copying #{source_path} to #{destination_path}") end end
Creates a new container. Returns true if the container was
created, false if it already existed.
# File lib/att/swift.rb, line 199 def create_container container authenticate uri = @storage_uri + container res = put uri case res when Net::HTTPCreated then true when Net::HTTPSuccess then false else raise Error.new(res, 'error creating container') end end
Deletes container. Returns true if the container existed,
false if not. Raises an exception otherwise.
# File lib/att/swift.rb, line 220 def delete_container container authenticate uri = @storage_uri + container res = delete uri case res when Net::HTTPNoContent then true when Net::HTTPNotFound then false when Net::HTTPConflict then raise Error.new(res, "container #{container} is not empty") else raise Error.new(res, "error deleting container") end end
Deletes object from container Returns true if
the object existed, false if not. Raises an exception otherwise.
# File lib/att/swift.rb, line 243 def delete_object container, object authenticate uri = @storage_uri + "#{container}/#{object}" res = delete uri case res when Net::HTTPNoContent then true when Net::HTTPNotFound then false else raise Error.new(res, "error deleting #{object} in #{container}") end end
Retrieves information for object in container
including metadata. Example result:
{
'metadata' => {
'meat' => 'Bacon',
},
'content-length' => '3072',
'content-type' => 'application/octet-stream',
'etag' => 'a2a5648d09a83c6a85ddd62e4a22309c',
'last-modified' => 'Wed, 22 Aug 2012 22:51:28 GMT',
}
# File lib/att/swift.rb, line 345 def object_info container, object authenticate uri = @storage_uri + "#{container}/#{object}" res = head uri case res when Net::HTTPSuccess then metadata = {} info = { 'metadata' => metadata } res.each do |name, value| case name when /^accept/, 'connection', 'date' then next when /^x-object-meta-(.*)/ then metadata[$1] = value when /^x-/ then next else info[name] = value end end info when Net::HTTPNotFound then nil else raise Error.new(res, "error retrieving metadata for #{object} in #{container}") end end
Retrieves metadata for object in container. See
also object_info. Example result:
'metadata' => { 'meat' => 'Bacon', }
# File lib/att/swift.rb, line 387 def object_metadata container, object info = object_info container, object return unless info info['metadata'] end
Lists objects in container. marker lists objects
after the given name, limit restricts the object listing to
that many items.
Example output:
[{"name"=>"test-0",
"hash"=>"b1946ac92492d2347c6235b4d2611184",
"bytes"=>6,
"content_type"=>"application/octet-stream",
"last_modified"=>"2012-08-22T19:24:03.102100"},
{"name"=>"test-1",
"hash"=>"802eeebb01b647913806a870cbb5394a",
"bytes"=>52707,
"content_type"=>"application/octet-stream",
"last_modified"=>"2012-08-22T19:32:24.474980"},
{"name"=>"test-2",
"hash"=>"90affbd9a1954ec9ff029b7ad7183a16",
"bytes"=>5,
"content_type"=>"application/octet-stream",
"last_modified"=>"2012-08-22T21:10:05.258080"}]
# File lib/att/swift.rb, line 311 def objects container, marker = nil, limit = nil authenticate params = { 'format' => 'json' } params['marker'] = marker if marker params['limit'] = limit if limit uri = @storage_uri + "#{container}?#{escape_params params}" res = get uri case res when Net::HTTPSuccess then JSON.parse res.body else raise Error.new(res, "error retrieving object list in #{container}") end end
Like objects, but only retrieves
limit objects at a time from the swift server and yields the
object information to the given block. marker may be used to
start pagination after a particular entry.
swift.paginate_objects 'container' do |object_info| p object_info.name end
# File lib/att/swift.rb, line 404 def paginate_objects container, marker = nil, limit = 1_000 return enum_for __method__, container, marker, limit unless block_given? chunk_objects container, marker, limit do |chunk| chunk.each do |object_info| yield object_info end end self end
Reads object from container. Unless the response
is HTTP 200 OK an exception is raised.
If no block is given the response body is read, checked for a matching MD5 checksum and returned.
If a block is given the response is yielded for custom handling and no MD5 checksum is calculated.
Example:
swift.read_object 'test-container', 'test-object' do |res| open destination, 'wb' do |io| res.read_body do |chunk| io.write chunk end end end
# File lib/att/swift.rb, line 450 def read_object container, object authenticate uri = @storage_uri + "#{container}/#{object}" get uri do |res| case res when Net::HTTPOK then if block_given? then yield res else body = res.body if res['ETag'] != Digest::MD5.hexdigest(body) then raise Error.new(res, "checksum retrieving object #{object} in #{container}") end body end when Net::HTTPNotFound then raise Error.new(res, "object #{object} in #{container} not found") else raise Error.new(res, "error retrieving object #{object} in #{container}") end end end
Sets the metadata for object in container.
Existing metadata will be overwritten.
# File lib/att/swift.rb, line 493 def set_object_metadata container, object, metadata = {} authenticate uri = @storage_uri + "#{container}/#{object}" headers = {} metadata.each do |name, value| headers["X-Object-Meta-#{name}"] = value end res = post uri, headers case res when Net::HTTPSuccess then true else raise Error.new(res, "error setting metadata on #{object} in #{container}") end end
Writes to object in container.
content may be a String or IO-like object. If
content is a String the response ETag is checked against the
MD5 hash of the local copy.
A block may be given instead of content. You will be given a
pipe you can write the content to:
r = Random.new swift.write_object 'test-container', 'test-object' do |io| r.rand(10).times do io.write r.bytes r.rand 10_000 end end
In all cases the response ETag is returned when the content was successfully uploaded.
# File lib/att/swift.rb, line 534 def write_object container, object, content = nil raise ArgumentError, 'provide block or content' if content and block_given? if block_given? then content, write = IO.pipe Thread.start do begin yield write ensure write.close end end end authenticate uri = @storage_uri + "#{container}/#{object}" req = Net::HTTP::Put.new uri.path case content when String then req.body = content req['ETag'] = Digest::MD5.hexdigest content else req['Transfer-Encoding'] = 'chunked' req.body_stream = content end req.content_type = 'application/octet-stream' res = @http.request uri, req case res when Net::HTTPCreated then res['ETag'] else raise Error.new(res, "error creating object #{object} in container #{container}") end end