Question

I’m trying to parse 3 arguments (this works) and request a file that has a bunch of txt and URLs in it. I’m trying to put each URL into an array I have try with some regexp with no luck and also with scan(URI.regexp) et.c and cant see to figure what I’m doing wrong.

#!/usr/bin/env ruby
require 'uri'
require 'open-uri'

HOST=ARGV[0]
ID=ARGV[1]
VERSION=ARGV[2]
MYLINKS = Array.new

file = open("http://#{HOST}/v1/dc/manifest/#{ID}/#{VERSION}").read

file.each_line do |line|
    #puts "doing #{line}" # this works..
    MYLINKS << URI.extract(line, ['http', 'https'])
end

PS: the file is a JSON file. This is all working on a Bash script but I’m migrating it to Ruby. In the Bash script I download the file to /tmp then I parse it with awk/tr etc.

Answer 1

With an API client like HTTParty you don’t really have to know JSON, because responses are parsed into a data structure you can extract. You can start with a simple class:

require 'httparty'

class MyApi
  include HTTParty
  format :json
end

Then you can do the following:

response = MyApi.get("http://#{HOST}/v1/dc/manifest/#{ID}/#{VERSION}")

and the response will be parsed into a hash structure that you’ll be able to pull your URLs from in a robust way.

If your API requires authentication or posting data, it is easy to add. There are several examples in the gem itself, so you can see how various things are done.