Ruby Read Large files from the Network and write to File

If you are writing a large file to disk using the traditional way (open-uri), you will notice that the memory usage spikes up just before writing the file to disk.

The workaround to this is to use the HTTP.start method and write chunks at a time to disk as they are received.

Net::HTTP.start(end_point, { :use_ssl =>true }) do |http|
   http.request_get(resource) do |response|
    open filename(date), 'w' do |io|
      response.read_body do |chunk|
        io.write chunk
      end
    end
  end
end