Question

I’ve got a string that I want to read line-by-line, but I also need to have the line delimiter character, which StringReader.ReadLine unfortunately trims (unlike in ruby where it is kept). What is the fastest and most robust way to accomplish this?

Alternatives I’ve been thinking about:

• Reading the input character-by-character and checking for the line delimiter each time
• Using RegExp.Split with a positive lookahead

Alternatively I only care about the line delimiter because I need to know the actual position in the string, and the delimiter can be either one or tho character long. Therefore if I could get back the actual position of the cursor within the string would be also good, but StringReader doesn’t have this feature.

EDIT: here is my current implementation. End-of-file is designated by returning an empty string.

StringBuilder line = new StringBuilder();
int r = _input.Read();
while (r >= 0)
{
  char c = Convert.ToChar(r);
  line.Append(c);
  if (c == '\n') break;
  if (c == '\r')
  {
    int peek = _input.Peek();
    if (peek == -1) break;
    if (Convert.ToChar(peek) != '\n') break;
  }
  r = _input.Read();
}
return line.ToString();

Answer 1

Are you concerned about inconsistencies between files (i.e. coming from Unix/Mac vs. Windows), or within files?

One very easy optimization if you know that individual files are consistent with themselves would be to only read the first line character-by-character and figure out what the delimiter is. Then determining the exact position of any other line would be simple math.

Failing that, I think I would go the character-by-character route. A regex seems too “clever.” This sounds like a complex function and I think the most important thing would be to make it easy to write, read, understand, and most importantly debug.

---

There’s another way to do this, which would be more efficient if your data source was a stream. Unfortunately it’s not, as referenced in your comment, so you would have to create one first; however, I’ll include the solution anyway, it might give you some inspiration:

public IEnumerable<int> GetLineStartIndices(string s)
{
    yield return 0;
    byte[] chars = Encoding.UTF8.GetBytes(s);
    using (MemoryStream stream = new MemoryStream(chars))
    {
        using (StreamReader reader = new StreamReader(stream, Encoding.UTF8))
        {
            while (reader.ReadLine() != null)
            {
                yield return stream.Position;
            }
        }
    }
}

This will give you back the start position of each new line. Obviously you can tweak this to do whatever else you need, i.e. do something else with the actual lines you read.

Just note that this has to make a copy of the string to create the byte array, so it’s really not suitable for very large strings. It’s a bit nicer than the char-by-char approach though, less bug-prone, so perhaps worth considering if the strings are not megabytes-long.