Question

lets say I have a text file with lines as such:

[4/20/11 17:07:12:875 CEST] 00000059 FfdcProvider  W com.test.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:/Prgs/testing/WebSphere/AppServer/profiles/ProcCtr01/logs/ffdc/server1_3d203d20_11.04.20_17.07.12.8755227341908890183253.txt com.test.testserver.management.cmdframework.CmdNotificationListener 134
[4/20/11 17:07:27:609 CEST] 0000005d wle           E   CWLLG2229E: An exception occurred in an EJB call.  Error: Snapshot with ID Snapshot.8fdaaf3f-ce3f-426e-9347-3ac7e8a3863e not found.
                             com.lombardisoftware.core.TeamWorksException: Snapshot with ID Snapshot.8fdaaf3f-ce3f-426e-9347-3ac7e8a3863e not found.
   at com.lombardisoftware.server.ejb.persistence.CommonDAO.assertNotNull(CommonDAO.java:70)

Is there anyway to easily import a data source such as this into protovis, if not what would the easiest way to parse this into a JSON format. For example for the first entry might be parsed like so:

[
  {
 "Date": "4/20/11 17:07:12:875 CEST",
 "Status": "00000059",
 "Msg": "FfdcProvider  W com.test.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I",
 },
]

Thanks, David

Answer 1

Protovis itself doesn’t offer any utilities for parsing text files, so your options are:

• Use Javascript to parse the text into an object, most likely using regex.
• Pre-process the text using the text-parsing language or utility of your choice, exporting a JSON file.

Which you choose depends on several factors:

• Is the data somewhat static, or are you going to be running this on a new or dynamic file each time you look at it? With static data, it might be easiest to pre-process; with dynamic data, this may add an annoying extra step.

• How much data do you have? Parsing a 20K text file in Javascript is totally fine; parsing a 2MB file will be really slow, and will cause the browser to hang while it’s working (unless you use Workers).

• If there’s a lot of processing involved, would you rather put that load on the server (by using a server-side script for pre-processing) or on the client (by doing it in the browser)?

If you wanted to do this in Javascript, based on the sample you provided, you might do something like this:

// Assumes var text = 'your text';
// use the utility of your choice to load your text file into the
// variable (e.g. jQuery.get()), or just paste it in.
var lines = text.split(/[\r\n\f]+/),
    // regex to match your log entry beginning
    patt = /^\[(\d\d?\/\d\d?\/\d\d? \d\d:\d\d:\d\d:\d{3} [A-Z]+)\] (\d{8})/,
    items = [],
    currentItem;

// loop through the lines in the file
lines.forEach(function(line) {
    // look for the beginning of a log entry
    var initialData = line.match(patt);
    if (initialData) {
        // start a new item, using the captured matches
        currentItem = {
            Date: initialData[1],
            Status: initialData[2],
            Msg: line.substr(initialData[0].length + 1)
        }
        items.push(currentItem);
    } else {
        // this is a continuation of the last item
        currentItem.Msg += "\n" + line;  
    }
});

// items now contains an array of objects with your data