I have a PHP file (feature on the site) – that allows User to import data from his account on another site.
I also need to copy LOTs of images from there every time he wants to import data.
E.g. 500 pics, 300-500Kb each is minumum. Expect this number easily to be 2000 images for one user.
The steps for each picture are:
- Get the picture URL
- Make image from URL (by using imagecreatefromjpeg and others)
- Save it on my server (using functions like imagejpeg, imagepng etc)
It already takes VERY long to execute this code (over 8 mins).
I realize it’s lots of data, but is there another possible way to do it?
Maybe run copying on background, or copy many pics simultaneously.
Just want to know if there is any technology that is designed specially for this, and I don’t know about it.
Or there is no other way except to outsource work with images to some image hosting server and only keep thumbnails.
Thanks.
There’s not a lot of information to go on here. What OS’s are being used? How “remote” is the source site? What format are the images already in?
If the other site is remote (i.e., a different hosting company), the main problem you will have is the speed at which the source server can pipe the data down to your machine.
One big question though is “what format are the images in at the moment?”. If the images are already JPEGs then retrieving and then converting into JPEG again will decrease the quality (albeit slightly). A better thing to do would be to just copy the image file directly. This will remove the time taken for your PHP app to re-encode a JPEG. Ask yourself – do you really need to convert the images?
Depending on what OS commands you have available, you would probably be better off calling applications which handle the transfer (e.g.
wgetin Linux). I’ve usedwgetto retrieve files from a remote server to a local one and it’s not too difficult to get running.Remember – the more steps you have in your transfer, the longer it is going to take. At the moment, you have:
all handled by PHP (probably going from slowest to quickest)
Does the source host provide an archive or export facility for customers? If so, could that be utilised to transfer files en masse?
Taking as much out of PHP as possible will make the process quicker. Calling system functions (e.g.,
wget,ftp,ssh,imagemagicketc) would make things quicker still (outside of PHP and Apache)