Question

// Calculating term frequency
int filename = 11;
String[] fileName = new String[filename];
int a = 0;
int totalCount = 0;
int wordCount = 0;


// Count inverse document frequency

System.out.println("Please enter the required word  :");
Scanner scan2 = new Scanner(System.in);
String word2 = scan2.nextLine();
String[] array2 = word2.split(" ");
int numofDoc;

for (int b = 0; b < array2.length; b++) {

    numofDoc = 0;

    for (int i = 0; i < filename; i++) {

        try {

            BufferedReader in = new BufferedReader(new FileReader(
                           "C:\\Users\\user\\fypworkspace\\TextRenderer\\abc"
                           + i + ".txt"));

            int matchedWord = 0;

            Scanner s2 = new Scanner(in);

            {

                while (s2.hasNext()) {
                    if (s2.next().equals(array2[b]))
                        matchedWord++;
                }

            }
            if (matchedWord > 0)
                numofDoc++;

        } catch (IOException e) {
            System.out.println("File not found.");
        }

    }
    System.out.println(array2[b]
                       + " --> This number of files that contain the term  "
                       + numofDoc);


    //calculate TF-IDF
    for (a = 0; a < filename; a++) {

        try {
            System.out.println("The word inputted : " + word2);
            File file =
                new File("C:\\Users\\user\\fypworkspace\\TextRenderer\\abc"
                         + a + ".txt");
            System.out.println(" _________________");

            System.out.print("| File = abc" + a + ".txt | \t\t \n");

            for (int i = 0; i < array2.length; i++) {

                totalCount = 0;
                wordCount = 0;

                Scanner s = new Scanner(file);
                {
                    while (s.hasNext()) {
                        totalCount++;
                        if (s.next().equals(array2[i]))
                            wordCount++;

                    }

                    System.out.print(array2[i] + " --> Word count =  "
                                     + "\t\t " + "|" + wordCount + "|");
                    System.out.print("  Total count = " + "\t\t " + "|"
                                     + totalCount + "|");
                    System.out.printf("  Term Frequency =  | %8.4f |",
                                      (double) wordCount / totalCount);

                    System.out.println("\t ");

                    double inverseTF = Math.log10((float) numDoc / numofDoc);
                    System.out.println("    --> IDF " +  inverseTF );

                    double TFIDF = (((double) wordCount / totalCount) * inverseTF );
                    System.out.println("    --> TF/IDF " + TFIDF);
                }
            }
        } catch (FileNotFoundException e) { 
            System.out.println("File is not found");
        }
    }
}

When i input a string, lets say ‘how’, the code will search for the number of files that contain the string ‘how’.

For example output :

The number of files containing 'how' is 5.

Then the code will proceed to calculate the term frequency-inverse document frequency.

When i input 3 strings, such as ‘how are you’.

The output will only display for only the string ‘how’.

Example output :

Please enter the required word  :
you

you --> This number of files that contain the term  6

The word inputted : you

 _________________
| File = abc0.txt |          
you --> Word count =         |3|  Total count =          |150|  Term Frequency =  |   0.0200 |   
    --> IDF 0.2632414441876607
    --> TF/IDF 0.005264828883753215

The word inputted : you

If i input 3 strings : ‘how are you’

Please enter the required word  :
how are you
how --> This number of files that contain the term  6

<— It will only process first string which is ‘how’

The word inputted : how are you
 _________________
| File = abc0.txt |          
how --> Word count =         |0|  Total count =          |150|  Term Frequency =  |   0.0000 |   
    --> IDF Infinity
    --> TF/IDF NaN

are --> Word count =         |0|  Total count =          |150|  Term Frequency =  |   0.0000 |   
    --> IDF Infinity
    --> TF/IDF NaN

you --> Word count =         |3|  Total count =          |150|  Term Frequency =  |   0.0200 |   
    --> IDF Infinity
    --> TF/IDF Infinity

Then the rest of the string will only use ONE number of files which is 0. Each string suppose to have their own individual NUMBER OF FILES.

How to make the code to receive 3 different NUMBER OF FILES ?

Answer 1

In order to count the number of documents per searchterm, you could use an int array to keep the counts:

String[] array2 = word2.split(" ");
int[] numofDoc = new int[array2.length];

for (int b = 0; b < array2.length; b++) {

    numofDoc[b] = 0;

use the array element when counting:

            if (matchedWord > 0) {
                numofDoc[b]++;
            }

and later use the array elements to calculate:

            double inverseTF = Math.log10((float) numDoc / numofDoc[i]);