I am writing a program that reads some IDs from a list, figures out different URLs from them, and saves the images to my C: drive.
The image URLs work if I navigate to them in my browser. Additionally, if I try URLs to images from a different server this program works entirely. The issue is when I try to connect to specific images at this URL it does not seem to work. The strange thing is when I ping the URL I get 0% packet loss. Additionally, my browser is not using special proxy settings that would cause it to work over the Java program.
What could be the cause of the below error/output?
Job started ...
1) AC_0A47_EXT1.jpg 2/0/47/307/398/HA47H01_H.jpg
Exception #3 java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at pullVFMImagesFromVFMURLs.writeImage(pullVFMImagesFromVFMURLs.java:99)
at pullVFMImagesFromVFMURLs.pullvfmimagesfromvfmurls(pullVFMImagesFromVFMURLs.java:48)
at pullVFMImagesFromVFMURLs.main(pullVFMImagesFromVFMURLs.java:21)
More info:
The ID list (MissingImagez.txt) is as follows:
AC_0A47_EXT1.jpg|2/0/47/307/398/xxx.jpg
AC_0C09_EXT1.jpg|3/0/44/130/589/xyz.jpg
AC_0C16_RM1.jpg|3/0/44/602/895/zzz.jpg
AC_0C17_BAR1.jpg|3/0/45/284/10/www.jpg
and the corresponding code is as follows:
public static void main(String[] args)
{
pullImagesFromURLs processor = new pullImagesFromURLs();
String passParam = null;
for (int i = 0; i < args.length; i++)
passParam = (args[i].toString().toUpperCase());
processor.pullvfmimagesfromvfmurls(passParam);
}
public void pullvfmimagesfromvfmurls(String passParam)
{
System.out.println("Job started ...");
try
{
boolean bOK = false;
String sOrigName = "";
String sURL = "";
int iCount = 0;
int iInt = 0;
try
{
BufferedReader in = new BufferedReader(new FileReader("/tmp/missingImagez.txt"));
String str;
while ((str = in.readLine()) != null)
{
iInt = str.indexOf("|");
sOrigName = str.substring(0,iInt);
sURL = str.substring(iInt+1);
iCount ++;
System.out.println(iCount + ") " + sOrigName + " " + sURL);
bOK = writeImage(sOrigName, sURL);
}
try
{
if (in != null)
in.close();
}
catch (Exception e)
{
}
}
catch (IOException e)
{
System.out.println("Exception #1 " + e);
}
}
catch (Exception e)
{
System.out.println("Exception #2 " + e);
System.exit(-666);
}
System.out.println("Job completed");
System.exit(-666);
}
private static boolean writeImage(String origName, String vfmURL)
{
boolean bOK = true;
String sPath = "C:/images/img2754/";
try
{
final String vfmURLPrefix = new String("http://www.blah.blah2.com/imageRepo/");
BufferedInputStream stream = null;
FileOutputStream destination = null;
URLConnection urlc = null;
int readSize = 1024 * 50;
int bytes_read;
URL url = null;
url = new URL(vfmURLPrefix + vfmURL);
urlc = url.openConnection();
urlc.setDoOutput(true);
stream = new BufferedInputStream(url.openStream());
System.out.println(url.toString());
destination = new FileOutputStream(sPath+origName);
byte[] buffer = new byte[readSize];
while(true)
{
bytes_read = stream.read(buffer);
if (bytes_read == -1) {break;}
destination.write(buffer, 0, bytes_read);
stream.close();
}
if(stream != null)
stream.close();
if(destination != null)
destination.close();
}
catch(Exception e)
{
System.out.println("Exception #3 " + e);
}
return bOK;
}
Tried using Apache HttpComponents. Get the following output:
Okay — I tried that…now I get:
Exception #3 org.apache.http.conn.HttpHostConnectException: Connection to http://www.blah.blah2.com/ refused.
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at pullVFMImagesFromVFMURLs.writeImage(pullVFMImagesFromVFMURLs.java:100)
at pullVFMImagesFromVFMURLs.pullvfmimagesfromvfmurls(pullVFMImagesFromVFMURLs.java:55)
at pullVFMImagesFromVFMURLs.main(pullVFMImagesFromVFMURLs.java:28)
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
... 9 more
EDIT 2
Your code has an error in the read loop. InputStream.read does not necessary return the entire byte stream being received. (It might not all be available yet.) The way you have coded it, the input stream is going to often be prematurely closed. That results in a different error (“stream closed”) than what you have posted though.
In any case, after fixing your loop, I was able to run your code to download this image without any problems.
Original
As an aside, your code is a kind-of confusing. In particular, your code creates a URLConnection, but opens the stream directly from the URL. The URLConnection is basically unused. It’s also confusing because you call setDoOutput, but not setDoInput. I would rewrite your code more like this:
I doubt that will solve your current problem though. To debug your current problem, I would recommend using Firebug or Chrome’s Developer Tools to sniff the network connection/request/response when you request the same image URL in your browser. You might find unexpected redirects, status codes, cookies, or similar. With that extra knowledge in hand, you then might try casting the connection to an HttpURLConnection so that you have more options available for customizing the request.