I have a model which represents a document, the title, author, etc is scraped from a web table, but the content needs to be pulled from a PDF which is time and resource intensive. This seems like a logical place to used delayed_jobs but it always fails and so I’m wondering if there is a limitation I am missing or a GCE
#working code
doc = Document.new
doc.title = html_table.row[i].title
doc.author = html_table.row[i].author
doc.link = html_table.row[i].link
doc.load_pdf_text # instance method which used the link extracted above to dowload a pdf and strip the text
doc.save
The above code works without any changes. I thought that delayed-job allowed replacing the direct method call with a send_later call and continuing
#broken code
#same as above except
doc.send_later(:load_pdf_text)
This code will always fail jobs and it will complain of accessing private methods which are part of the model, for example, the URL is needed to download the PDF. Any pointers would be greatly appreciated.
Added:
method definition (link and summary is a database backed attribute of Document model)
def load_pdf_text
tmpfilename = "#{RAILS_ROOT}/tmp/tmp_"+Time.now.to_s.gsub(/ |-/,"_")
Kernel.system("curl -s #{link} > #{tmpfilename}") # this is line 224
a=""
IO.popen("pdftotext -raw -l 1 #{tmpfilename} -"){ |pipe| a = pipe.read }
Kernel.system("rm #{tmpfilename}")
summary = a
save
end
Error message
--- !ruby/struct:Delayed::PerformableMethod
object: AR:Memo:4089
method: :load_pdf_text
args: []
| Attempt to call private method
/Library/Ruby/Gems/1.8/gems/activerecord-2.3.2/lib/active_record/attribute_methods.rb:236:in `method_missing'
/Users/naven87/test/memos/app/models/memo.rb:224:in `load_pdf_text'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/performable_method.rb:23:in `send'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/performable_method.rb:23:in `perform'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/job.rb:215:in `invoke_job'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/job.rb:93:in `run_with_lock'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:62:in `timeout'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/job.rb:93:in `run_with_lock'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/job.rb:92:in `run_with_lock'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/job.rb:153:in `reserve_and_run_one_job'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/job.rb:152:in `each'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/job.rb:152:in `reserve_and_run_one_job'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/job.rb:199:in `work_off'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/job.rb:198:in `times'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/job.rb:198:in `work_off'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/worker.rb:28:in `start'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/worker.rb:27:in `start'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/worker.rb:24:in `loop'
/Library/Ruby/Gems/1.8/gems/delayed_job-1.8.4/lib/delayed/worker.rb:24:in `start'
/Users/naven87/.gem/ruby/1.8/gems/collectiveidea-delayed_job-1.8.2/lib/delayed/tasks.rb:13
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `execute'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `execute'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:597:in `invoke_with_call_chain'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/monitor.rb:242:in `synchronize'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2051:in `invoke_task'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `each'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2023:in `top_level'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2001:in `run'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in `run'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/bin/rake:31
/usr/bin/rake:19:in `load'
/usr/bin/rake:19
Based on further investigation it appears that when delayed job serializes the object and the deserializes to actually send the message it is no longer in the same scope and it cannot access any private instance methods of the object. This is confusing because it you weren’t running it through delayed job, it would work find, but once passed to delayed job it no longer acts as the object running methods on itself but rather sending methods to the object whcih creates a different set of permissions. you need to either make the instance method public or provide a public mechanism that delayed job can use to do any manipulations that would oterhwise be private.