Question

I am configuring Jenkins job for uploading files from Subversion to SFTP. Publish Over SSH is doing quite well, but it uploads all files on every build.

For some projects we have thousands of files and upload cost more than 1 hour so that’s not an option.

Can anyone suggest a way to upload only files changed in last revision?

Answer 1

You can use Subversion Plugin to poll for SVN changes and run a job that does nothing in particular. Let’s call it YourPollingJob. Then call Jenkins API via http like this:

http://[jenkins_server]/api/xml?depth=2&xpath=/hudson/job[name='YourPollingJob']/build[id='BUILD_ID']/changeSet

where BUILD_ID is the actual build id of the job you’ve just run (the usual format is something like 2012-02-21_16-15-49). Examine the result. Note that all the information about the files that have changed since the previous build is there – you just need to parse it out.

So now you can do the following: YourPollingJob will call another job – let’s call it CopyJob – and pass to it its BUILD_ID as a parameter (via the Parameterized Trigger Plugin; make sure to call CopyJob as a post-build step, not as a build step). CopyJob will then query Jenkins via http (as above, the best way is to do it via wget), parse the results, and do the copy.

You can do it all in one job, but it’s a bit more complicated and hard to debug.

Also, it is probably prudent to copy the whole repository once a day (nightly).