I’m trying to implement some simple caching into my application and I’m having some issues.
Basically my code scrapes a webpage, gets some regex matches and puts them into an ArrayList. This takes a while to do (a few seconds, but this happens frequently) so I’m trying to implement a simple caching system which checks when the cache file was modified, if this is older than a week, refresh cache.
My problem lies with storing the ArrayList, can anyone help with this?
I originally have (which works)
private List<String> getVideoIDS(final String url2) {
if (android.os.Environment.getDataDirectory().equals(android.os.Environment.MEDIA_MOUNTED)) //android.os.Environment.getExternalStorageState().equals(android.os.Environment.MEDIA_MOUNTED))
cacheDir=new File(android.os.Environment.getDataDirectory(),"PL" + url.getQuery());
else
cacheDir=mContext.getCacheDir();
if(!cacheDir.exists())
cacheDir.mkdirs();
File f = new File(cacheDir, "PL"+url.getQuery());
if (f.lastModified() > 604800000 || f.exists() == false) {
String expr = "(?<=v=)[a-zA-Z0-9-_]{11}(?=&)|(?<=[0-9]/)[^&\n]{11}|(?<=v=)[^&\n]{11}";
int count = 0;
Pattern patt = Pattern.compile(expr,
Pattern.DOTALL | Pattern.UNIX_LINES | Pattern.CASE_INSENSITIVE);
try {
url = new URL(url2);
} catch (MalformedURLException e) {
e.printStackTrace();
}
try {
m = patt.matcher(getURLContent(url));
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
while (m.find()) {
if ("default.jpg".equals(m.group(0)) || "videos?view".equals(m.group(0))) {
//Do nothing
} else {
allMatches.add(m.group(0));
count++;
Log.i("", m.group(0));
}
}
//Remove duplicates
List<String> noDupes = new ArrayList<String>();
Iterator iterator = allMatches.iterator();
while (iterator.hasNext())
{
String o = (String) iterator.next();
if(!noDupes.contains(o)) noDupes.add(o);
}
allMatches = noDupes;
return allMatches; }
Now what I’m trying to modify this to be is this (in theory, as it currently doesn’t work)
private List<String> getVideoIDS(final String url2) {
if (android.os.Environment.getDataDirectory().equals(android.os.Environment.MEDIA_MOUNTED)) //android.os.Environment.getExternalStorageState().equals(android.os.Environment.MEDIA_MOUNTED))
cacheDir=new File(android.os.Environment.getDataDirectory(),"PL" + url.getQuery());
else
cacheDir=mContext.getCacheDir();
if(!cacheDir.exists())
cacheDir.mkdirs();
File f = new File(cacheDir, "PL"+url.getQuery());
if (f.lastModified() > 604800000 || f.exists() == false) {
String expr = "(?<=v=)[a-zA-Z0-9-_]{11}(?=&)|(?<=[0-9]/)[^&\n]{11}|(?<=v=)[^&\n]{11}";
int count = 0;
Pattern patt = Pattern.compile(expr,
Pattern.DOTALL | Pattern.UNIX_LINES | Pattern.CASE_INSENSITIVE);
try {
url = new URL(url2);
} catch (MalformedURLException e) {
e.printStackTrace();
}
try {
m = patt.matcher(getURLContent(url));
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
while (m.find()) {
if ("default.jpg".equals(m.group(0)) || "videos?view".equals(m.group(0))) {
//Do nothing
} else {
allMatches.add(m.group(0));
count++;
Log.i("", m.group(0));
}
}
//Remove duplicates
List<String> noDupes = new ArrayList<String>();
Iterator iterator = allMatches.iterator();
while (iterator.hasNext())
{
String o = (String) iterator.next();
if(!noDupes.contains(o)) noDupes.add(o);
}
allMatches = noDupes;
ObjectOutputStream save = null;
FileOutputStream saveFile = null;
try {
saveFile = new FileOutputStream(cacheDir + "PL"+url.getQuery());
} catch (FileNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
// Create an ObjectOutputStream to put objects into save file.
try {
save = new ObjectOutputStream(saveFile);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
try {
save.writeObject(allMatches);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
save.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return allMatches;}
else {
try{
// Open file to read from, named SavedObjects.sav.
FileInputStream saveFile = new FileInputStream(cacheDir + "PL"+url.getQuery());
// Create an ObjectInputStream to get objects from save file.
final ObjectInputStream save = new ObjectInputStream(saveFile);
// Now we do the restore.
// readObject() returns a generic Object, we cast those back
// into their original class type.
// For primitive types, use the corresponding reference class.
allMatches = (ArrayList) save.readObject();
// Close the file.
save.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (StreamCorruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
// This also closes saveFile.
}
} return allMatches;
}
Note: I’m trying to save multiple files as this method is called for various different categories. That should be handled with calling the file a different filename based on whats open. I think I have implemented this correctly.
Stacktrace is as follows
04-03 17:41:20.846: E/AndroidRuntime(2438): java.lang.RuntimeException: Unable to start activity ComponentInfo{com.test.test/com.test.test.VideoGrid}: java.lang.NullPointerException
04-03 17:41:20.846: E/AndroidRuntime(2438): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2194)
04-03 17:41:20.846: E/AndroidRuntime(2438): at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2229)
04-03 17:41:20.846: E/AndroidRuntime(2438): at android.app.ActivityThread.access$600(ActivityThread.java:139)
04-03 17:41:20.846: E/AndroidRuntime(2438): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1261)
04-03 17:41:20.846: E/AndroidRuntime(2438): at android.os.Handler.dispatchMessage(Handler.java:99)
04-03 17:41:20.846: E/AndroidRuntime(2438): at android.os.Looper.loop(Looper.java:154)
04-03 17:41:20.846: E/AndroidRuntime(2438): at android.app.ActivityThread.main(ActivityThread.java:4944)
04-03 17:41:20.846: E/AndroidRuntime(2438): at java.lang.reflect.Method.invokeNative(Native Method)
04-03 17:41:20.846: E/AndroidRuntime(2438): at java.lang.reflect.Method.invoke(Method.java:511)
04-03 17:41:20.846: E/AndroidRuntime(2438): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:784)
04-03 17:41:20.846: E/AndroidRuntime(2438): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:551)
04-03 17:41:20.846: E/AndroidRuntime(2438): at dalvik.system.NativeStart.main(Native Method)
04-03 17:41:20.846: E/AndroidRuntime(2438): Caused by: java.lang.NullPointerException
04-03 17:41:20.846: E/AndroidRuntime(2438): at java.io.DataOutputStream.writeShort(DataOutputStream.java:192)
04-03 17:41:20.846: E/AndroidRuntime(2438): at java.io.ObjectOutputStream.writeStreamHeader(ObjectOutputStream.java:1815)
04-03 17:41:20.846: E/AndroidRuntime(2438): at java.io.ObjectOutputStream.<init>(ObjectOutputStream.java:279)
04-03 17:41:20.846: E/AndroidRuntime(2438): at com.test.test.VideoGrid.getVideoIDS(VideoGrid.java:204)
04-03 17:41:20.846: E/AndroidRuntime(2438): at com.test.test.VideoGrid.onCreate(VideoGrid.java:74)
04-03 17:41:20.846: E/AndroidRuntime(2438): at android.app.Activity.performCreate(Activity.java:4524)
04-03 17:41:20.846: E/AndroidRuntime(2438): at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1071)
04-03 17:41:20.846: E/AndroidRuntime(2438): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2150)
This
com.test.test.VideoGrid.getVideoIDS(VideoGrid.java:204)
Points to
save = new ObjectOutputStream(saveFile);
I have
android.permission.WRITE_EXTERNAL_STORAGE
In my manifest
I have tried changing the filename to something static (1) instead of my url.getQuery incase that was the problem – doesn’t change anything, exact same problem.
I feel I’m making a fundamental error here. Can anyone see where I’m going wrong with saving/reading the ArrayList?
It looks like your save file wasn’t found; but its hard to tell since your code is currently swallowing all of the exceptions. instead of :
Try logging your exceptions instead, since e.printStackTrace(); doesn’t do anything by default on android.
Anyway, that’s my theory – your file path is pointing to somewhere that can’t be written to for whatever reason (the docs have some possible explanations); saveFile then comes out of the exception as null, and then you pass the null parameter into the outputStream constructor