Question

Is there an easy way, preferably with a scripting language or a small tool that could be called through a batch file, to operate on a text file, mark an offset, and put everything after the offset into a new file?

I have a text file added to nightly, and I would like to make it so that the end of the file is marked, then after new data is added, only data between the offset and the end is processed. I can not do this with just strings or delimiters as it is blob data.

edit: The text file is created by running a ms access macro from a scheduled tasks, which exports the data as a csv file. In considering Patricks suggestion, I would like to know if it would be possible to add a wildcard such as the date to the filename, to always have a different file. This file will then be scp’d to a linux server, where it will be loaded into a mysql database.

Answer 1

Assuming you’re currently exporting the data from the Access database with a script already:

@echo OFF  :: Force a new line and add a marker; assuming your file is data.txt. @echo. >> data.txt @echo **MARKER** >> data.txt  :: Run your export here: these lines just simulate the export. @echo Test Line 1 >> data.txt @echo Test Line 2 >> data.txt  :: Find line number of last marker: for /f 'usebackq delims=:' %%I in (`findstr /N '**MARKER**' data.txt`) do (     set LAST_MARKER=%%I )  :: Get all the lines after the last marker for /f 'skip=%LAST_MARKER% tokens=*' %%L in (data.txt) do (     @echo %%L >> new_data.txt ) 

The output in new_data.txt will be:

Test Line 1
Test Line 2