Question

I am building a java application as a learning exercise. For now, I want to make an app that will take the contents in a folder and replicate it to my Amazon S3 bucket. By searching around, I found out that the best way to tell if 2 files are identical is to take the MD5 value.

How do I iteratively take the MD5 of each file in my bucket?

According to this link:
http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?ListingObjectKeysUsingJava.html

The ObjectListing class can list the object keys and (metadata?) of your files in the bucket. But, it doesn’t really specify how I can interact with the files. For example,
for my local files, I use guava and does something like this:

MessageDigest md = MessageDigest.getInstance("MD5");
byte[] md4 = Files.getDigest(localFile, md);

Looking at the AWS S3 API, you can use S3Object.getObjectContent() which will return an InputStream. Is that the best way to work with the objects directly on S3?

Finally, what is the best way to sync from my local folder to a bucket in S3? Any tips is welcomed.

Thanks!!!!

Answer 1

ObjectListing objectListing = s3.listObjects(
  new ListObjectsRequest().withBucketName(bucket));
List<S3ObjectSummary> l = objectListing.getObjectSummaries();

S3ObjectSummary has memeber called eTag which is md5 hash of the object