for example, if I want to implement a program that uses TF-IDF to extract keywords from articles, I have implemented the part of deactivating words. The contents of the current .txt file are shown in the following figure:
is a mixture of Chinese and English, the content specifications of the document are as follows: 1, each line of the document is an article; 2, according to the number of lines shown in the figure, there are a total of 15 articles.
now I want to extract the keywords of the top 10 TF-IDF scores in each article and write them into the file. How can I implement this program?
suppose the path of the file is now in the original text of D:\ delsw_ (under the root directory of the D disk).
I want to talk about the problems I encountered. My problems are: 1. I want to use the readline () method to read each line and then deal with it line by line, but I have tried a lot of programs that always write the wrong, readline () after each line is in the form of a string. I want to distinguish each word with a space, but I can"t do it. 2. I want to use the list list to store the contents of each line, and after processing one line, I will clear it and then process the next line, loop. The theory feels very simple, but it is really difficult for beginners to achieve, so ask for advice from all the great gods, thank you!
attach TF-IDF "s formula:
TF (word frequency) = the number of times the word appears in the article / the total number of words in the article
IDF (inverse document frequency) = log (total number of documents / number of documents containing the word + 1)
TF-IDF=TF*IDF
hope that God can help achieve it, thank you very much!