Question

I have an input stream with the following lines:

# <int> <int>
<some_data_type> <some_data_type> <some_data_type> ..... <some_data_type>
<some_data_type_1> <some_data_type_2> <some_data_type_3> <some_data_type_1> <some_data_type_2> <some_data_type_3> .... <some_data_type_1> <some_data_type_2> <some_data_type_3> 

In the above stream all three lines are different and have to be parsed differently. Currently,I am using a reading method as follows:

void reader( std::istream & is, DataStructure & d ){
  std::string line;
  getline(is,line);
  std::stringstream s(line);
  //parse line 1

  getline(is,line);
  std::stringstream line2(line);
  //parse line 2

  getline(is,line);
  std::stringstream line3(line);
  //parse line 3

 }

Now the idea is not to make use of std::stringstream at all, as a line can arbitarily large and we donot want to load everything into memory twice. So, it would be better if it was possible to read from the input stream directly into the user given datastructure d.

An idea is to make use of std::istream_iterator but unfortunately the different lines have different parsing needs. For example, in the last line, three elements from the stream together constitute a single data element.

The only idea that seems plausible to me at this moment is to handle the stream buffer directly. It would be great if anyone could recommend a better way of doing this.

NOTE: Cannot make use of a tertiary data structure like std::stringstream. It is essential to read from the stream directly into the user provided data structure.

EDIT: Please note we are only allowed a single pass over the file.

Answer 1

Now the idea is not to make use of std::stringstream at all, as a line
can arbitarily large and we donot want to load everything into memory
twice. So, it would be better if it was possible to read from the
input stream directly into the user given datastructure d.

Olaf explained the extraction operator above but then we have a new requirement:

This will only work for the first line, where it is known there is a
fixed number of elements.

and

(2) Unfortunately, I have no discriminator beyond my knowledge that each instance of the data
structure needs to be instantiated with information stored in three
different lines. All three lines have different lengths and different
data elements. Also, I cannot change the format.

plus

(3) All information is treated as unsigned integer.

Now the next issue is that we don’t know what the data structure actually is, so given what has come before it appears to be dynamic in some fashion. Because we can treat the data as unsigned int then we can use the extraction operator possibly, but read into a dynamic member:

vector<unsigned int> myUInts;
...
inFile >> currentUInt;
myUInts.push_back(currentUInt);

But then the issue of where to stop comes into play. Is it at the end of the first line, the third? If you need to read an arbitrary number of unsigned ints, whilst still checking for a new line then you will need to process white space as well:

inFile.unsetf(ios_base::skipws);

How you actually handle that is beyond what I can say at the moment without some clearer requirements. But I would guess it will be in the form:

 inFile >> myMember;
 char next = infile.peek()
 //skip whitespace and check for new line
 //Repeat until data structure filled, and repeat for each data structure.