Question

On a Linux system, I have a very large text file and I need to create a new text file which contains every line between the first and last of occurrence of a particular sessionId (those lines included).

I guess I probably need to use sed or something?

As a bonus, sometimes I won’t know which log file will contain the session trace. So a script that can work with regular expressions would be ideal. In this case I would expect the script to find the first file with the sessionId in it and then crop that file before exiting.

Example Log file looking for sessionId 1111-ABCD-1111-SOME-GUID :

line one containing other session id: 2222-ABCD-1111-SOME-GUID blaa blaa blaa
line two blaa blaa blaa
line three containing my session id: 1111-ABCD-1111-SOME-GUID blaa blaa blaa
line four containing other session id: 2222-ABCD-1111-SOME-GUID
line five blaa blaa blaa
line six containing other session id: 3333-ABCD-1111-SOME-GUID blaa blaa blaa
line seven containing other session id: 2222-ABCD-1111-SOME-GUID
line eight containing my session id: 1111-ABCD-1111-SOME-GUID blaa blaa blaa
line nine containing other session id: 3333-ABCD-1111-SOME-GUID
line ten containing my session id: 1111-ABCD-1111-SOME-GUID
line eleven
line twelve containing other session id: 3333-ABCD-1111-SOME-GUID blaa blaa blaa
line thirteen containing my session id: 1111-ABCD-1111-SOME-GUID
line fouteen blaa blaa blaa
line fifteen containing other session id: 3333-ABCD-1111-SOME-GUID blaa blaa blaa

output file would contain lines three to thirteen inclusive.

Answer 1

The following script will do all of what you asked, including the bonus. Put this script in the top-level directory that contains all the possible files with the ‘uid’ you want to crop. The script will recursively search this directory and crop all files that match and put the result in a new file with a .crp extension at the end (see example below). I took special consideration to make sure that this script will work with whatever filename you throw at it, whether it contain spaces or newlines or whatever in its name.

#!/bin/bash
uid="1111-ABCD-1111-SOME-GUID"

while IFS= read -r -d $'\0' file; do
    printf "%s\n" "?$uid?+1,\$d" "1,/$uid/-1d" "%p" | ex -s "$file" > "$file".crp
    echo "$file being cropped"
done < <(grep -lZR --exclude="${0#*/}" --exclude="*.crp" "$uid" .)

Result

$ ./uid.sh
./sample1.txt being cropped
./subdir/sample2.txt being cropped

$ cat ./sample1.txt.crp
line three containing my session id: 1111-ABCD-1111-SOME-GUID blaa blaa blaa
line four containing other session id: 2222-ABCD-1111-SOME-GUID
line five blaa blaa blaa
line six containing other session id: 3333-ABCD-1111-SOME-GUID blaa blaa blaa
line seven containing other session id: 2222-ABCD-1111-SOME-GUID
line eight containing my session id: 1111-ABCD-1111-SOME-GUID blaa blaa blaa
line nine containing other session id: 3333-ABCD-1111-SOME-GUID
line ten containing my session id: 1111-ABCD-1111-SOME-GUID
line eleven
line twelve containing other session id: 3333-ABCD-1111-SOME-GUID blaa blaa blaa
line thirteen containing my session id: 1111-ABCD-1111-SOME-GUID

$ cat ./subdir/sample2.txt.crp
line three containing my session id: 1111-ABCD-1111-SOME-GUID blaa blaa blaa
foo
bar
line eight containing my session id: 1111-ABCD-1111-SOME-GUID blaa blaa blaa
baz
line ten containing my session id: 1111-ABCD-1111-SOME-GUID

As you can see in the example above, my script found two files which matched, one of which was in a sub-directory below the top-level directory.