Question

I want to split a XML Like string to tokens in c# or sql.
for example
input string is like

<entry><AUTHOR>C. Qiao</AUTHOR> and <AUTHOR>R.Melhem</AUTHOR>, "<TITLE>Reducing Communication </TITLE>",<DATE>1995</DATE>. </entry>

and I want this output:

C       AUTHOR
.       AUTHOR
Qiao    AUTHOR
and 
R       AUTHOR
.       AUTHOR
Melhem  AUTHOR
,   
"
Reducing        TITLE
Communication   TITLE
"
,
1995    DATE
.

Answer 1

This is the first attempt on how to solve this problem, considering the following:
1. XML String will be valid (i.e. there’s not going to be any invalid chars between tags)
Like this:

string xml = @"<ENTRY><AUTHOR>C. Qiao</AUTHOR>
                                  <AUTHOR>R.Melhem</AUTHOR>
                                  <TITLE>Reducing Communication </TITLE>
                                  <DATE>1995</DATE>
                           </ENTRY>";

2. Splitting will be done by space ' '

string xml = @"<ENTRY><AUTHOR>C. Qiao</AUTHOR>
                              <AUTHOR>R.Melhem</AUTHOR>
                              <TITLE>Reducing Communication </TITLE>
                              <DATE>1995</DATE>
                       </ENTRY>";
        XElement doc = XElement.Parse(xml);
        foreach (XElement element in doc.Elements())
        {

            var values = element.Value.Split(' ');
            foreach (string value in values)
            {
                Console.WriteLine(element.Name + " " + value);
            }
        }

Will print out

AUTHOR C.
AUTHOR Qiao
AUTHOR R.Melhem
TITLE Reducing
TITLE Communication
TITLE
DATE 1995

EDIT:

Now, to split based on “.” and a space, the best idea is to use regex. Like this:

   var values = Regex.Split(element.Value, @"(\.| )");
        foreach (string value in values.Where(x=>!String.IsNullOrWhiteSpace(x)))
        {
            Console.WriteLine(element.Name + " " + value);
        }   

You can add more delimiters if you’d like. The following example will give you the following:

AUTHOR C
AUTHOR .
AUTHOR Qiao
AUTHOR R
AUTHOR .
AUTHOR Melhem
TITLE Reducing
TITLE Communication
DATE 1995

Edit2:
And here’s an example that works with your original string, it is most likely not the best approach, since it doesn’t have a correct ordering of tokens, but it should be pretty close:

 string xml = @" <entry>
                            <AUTHOR>C. Qiao</AUTHOR> 
                            and 
                            <AUTHOR>R.Melhem</AUTHOR>, 
                            ""<TITLE>Reducing Communication </TITLE>""
                           ,<DATE>1995</DATE>. 
                           </entry>";
            //Parse xml to XDocument
            XDocument doc = XDocument.Parse(xml);

            // Get first element (we only have one)
            XElement element = doc.Descendants().FirstOrDefault();

            //Create a copy of an element for use by child elements.
            XElement copyElement = new XElement(element);
            //Remove all child nodes from root leaving only text
            element.Elements().Remove();

            //Splitting based on the tokens specified
                var values = Regex.Split(element.Value, @"(\.| |\,|\"")");
                    foreach (string value in values.Where(x => !String.IsNullOrWhiteSpace(x)))
                    {
                        Console.WriteLine(value);
                    }
            //Getting children nodes and splitting the same way
            foreach (XElement elem in copyElement.Elements())
            {
                var val = Regex.Split(elem.Value, @"(\.| |\,|\"")");
                foreach (string value in val.Where(x => !String.IsNullOrWhiteSpace(x)))
                {
                    Console.WriteLine(value + " " + elem.Name);
                }
            }
            //You can try to play with DescendantsAndSelf 
            //to see if you can do it in single action and with order preserved.
            //foreach (XElement elem in element.DescendantsAndSelf())
            //{
            //    //....
            //}   

This will print out the following:

and
,
"
"
,
.
C AUTHOR
. AUTHOR
Qiao AUTHOR
R AUTHOR
. AUTHOR
Melhem AUTHOR
Reducing TITLE
Communication TITLE
1995 DATE