Analyzing Text Data

Choosing the right text mining method is crucial because it significantly impacts the quality of insights and information you can extract from your text data. Each method provides different insights and requires different amounts of data, training, and iteration. Before you search for data, it is essential that you:

1. identify the goals of your analysis
2. determine the method you will use to meet those goals
3. identify how much data you need for that method
4. develop a sampling plan to build a data set that accurately represents your object of study.

Starting with this information in mind will make your project go more quickly and smoothly, and help you overcome a lot of hurdles such as incomplete data, too much or too little data, or problems with access to data.

More Resources:

• Content Analysis Method and Examples, from Mailman School of Public Health, Columbia University
• Qualitative Research Methods Overview, from Northeastern University

Before you start collecting data, think about how much data you really need. New researchers in text analysis often want to collect every source mentioning their topic, but this is usually not the best approach. Collecting so much data takes a lot of time, uses many computational resources, often goes against platform terms of service, and doesn't necessarily improve analysis.

In text analysis, an essential idea is saturation, where adding more data doesn't significantly improve performance. Saturation is when the model has learned as much as it can from the available data, and no new patterns are themes are emerging with additional data. Researchers often use experimentation and learning curves to determine when saturation occurs; you can start by analyzing a small or mid-sized dataset and see what happens if you add more data.

Once you know your research question, the next step is to create a sampling plan. In text analysis, sampling means selecting a representative subset of data from a larger dataset for analysis. This subset, called the sample, aims to capture the diversity of sentiments in the overall dataset. The goal is to analyze this smaller portion to draw conclusions about the information in the entire dataset.

For example, in a large collection of customer reviews, sampling may involve randomly selecting a subset for sentiment analysis instead of analyzing every single review. This approach saves computational resources and time while still providing insights into the overall sentiment distribution of the entire dataset. It's crucial to ensure that the sample accurately reflects the diversity of sentiments in the complete dataset for valid and reliable generalizations.

Example Sampling Plans

Sampling plans for text analysis involve selecting a subset of text data for analysis rather than analyzing the entire dataset. Here are two common sampling plans for text analysis:

1. Random Sampling:

   • Description: Randomly select a subset of text documents from the entire dataset.
   • Process: Assign each document a unique identifier and use a random number generator to choose documents for inclusion in the sample.

2. Stratified Sampling:

   • Description: Divide the dataset into distinct strata or categories based on certain characteristics (e.g., product types, genres, age groups, race or ethnicity). Then, randomly sample from each stratum.
   • Process: Divide the dataset into strata, and within each stratum, use random sampling to select a representative subset.

Remember, the choice of sampling plan depends on the specific goals of the analysis and the characteristics of the dataset. Random sampling is straightforward and commonly used when there's no need to account for specific characteristics in the dataset. Stratified sampling is useful when the dataset has distinct groups, and you want to ensure representation from each group in the sample.

Exactly How Many Sources do I need?

Determining the amount of data needed for text analysis involves a balance between having enough data to train a reliable model and avoiding unnecessary computational costs. The ideal dataset size depends on several factors, including the complexity of the task, the diversity of the data, and the specific algorithms or models being used.

• Task Complexity: If you are doing a simple task, like sentiment analysis or basic text classification, a few dozen articles might be enough. More complex tasks, like language translation or summarization, often require datasets on the scale of tens of thousands to millions.
• Model Complexity: Simple models like Naive Bayes often perform well with smaller datasets, whereas complex models, such as deep learning models with many parameters, will require larger datasets for effective training.
• Data Diversity: Ensure that the dataset is diverse and representative of the various scenarios the model will encounter. A more diverse dataset can lead to a more robust and generalizable model. A large dataset that is not diverse will yield worse results than a smaller, more diverse dataset.
• Domain-Specific Considerations: Sometimes there is not a lot of data available, and it is okay to make do with what you have!

Start by taking a look at articles in your field that have done a similar analysis. What approaches did they take? You can also schedule an appointment with a Data Services Librarian to get you started.

More Readings on Sampling Plans for Text Analysis:

• How to Choose a Sample Size in Qualitative Research, from LinkedIn Learning (members of the GW community have free access to LinkedIn Learning using their GW email account)
• Sampling in Qualitative Research, from Saylor Academy
• Lowe, A., Norris, A. C., Farris, A. J., & Babbage, D. R. (2018). Quantifying Thematic Saturation in Qualitative Data Analysis. Field Methods, 30(3), 191-207. https://doi.org/10.1177/1525822X17749386

Word frequency analysis in text mining is a technique that involves counting how often each word appears in a given collection of text data, such as documents, articles, or web pages. It helps identify the most frequently occurring words and their frequencies.  This analysis is essential for understanding the importance and prevalence of words within the text, which can be used for tasks like identifying keywords, determining common themes, or detecting anomalies in a dataset. Word frequency analysis provides valuable insights into the structure and content of textual information, aiding in various text mining and natural language processing tasks.

Software for Word Frequency Analysis

Related Tools Available Online

Related Library Resources

Example Projects Using Word Frequency Analysis

• Robinson, J. S. and D. (n.d.). 3 Analyzing word and document frequency: Tf-idf | Text Mining with R. Retrieved November 21, 2023, from https://www.tidytextmining.com/tfidf.html

• Zhang, Z. (n.d.). Text Mining for Social and Behavioral Research Using R. Retrieved November 21, 2023, from https://books.psychstat.org/textmining/index.html

Machine learning for text analysis is a technology that teaches computers to understand and interpret written language by exposing them to examples. There are two types of machine learning for text analysis: supervised learning, in which a human helps to train the computer to detect patterns, and unsupervised learning, which enables computers to automatically categorize, analyze, and extract information from text without needing explicit programming.

One type of machine learning for text analysis is Natural Language Processing (NLP). NLP for text analysis is a field of artificial intelligence that involves the development and application of algorithms to automatically process, understand, and extract meaningful information from human language in textual form. NLP techniques are used to analyze and derive insights from large volumes of text data, enabling tasks such as sentiment analysis, named entity recognition, text classification, and language translation. The aim is to equip computers with the capability to comprehend and interpret written language, making it possible to automate various aspects of text-based information processing.

Software for Natural Language Processing

Related Resources Available Online

• [Large Language Models] LLMs in Scientific Research

Related Library Resources

Example Projects using Natural Language Processing

• Redd D, Workman TE, Shao Y, Cheng Y, Tekle S, Garvin JH, Brandt CA, Zeng-Treitler Q. Patient Dietary Supplements Use: Do Results from Natural Language Processing of Clinical Notes Agree with Survey Data? Medical Sciences. 2023; 11(2):37. https://doi.org/10.3390/medsci11020037
• Nguyen D, Liakata M, DeDeo S, Eisenstein J, Mimno D, Tromble R, Winters J. How We Do Things With Words: Analyzing Text as Social and Cultural Data. Front Artif Intell. 2020 Aug 25;3:62. doi: 10.3389/frai.2020.00062. 

Sentiment analysis is a method of analyzing text to determine whether the emotional tone or sentiment expressed in a piece of text is positive, negative, or neutral. Sentiment analysis is commonly used in businesses to gauge customer feedback, social media monitoring, and market research.

Software for Sentiment Analysis

Related Resources Available Online

• finnstats. (2021, May 16). Sentiment analysis in R | R-bloggers. https://www.r-bloggers.com/2021/05/sentiment-analysis-in-r-3/

Example Projects Using Sentiment Analysis

• Duong, V., Luo, J., Pham, P., Yang, T., & Wang, Y. (2020). The Ivory Tower Lost: How College Students Respond Differently than the General Public to the COVID-19 Pandemic. 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 126–130. https://doi.org/10.1109/ASONAM49781.2020.9381379
• Ali, R. H., Pinto, G., Lawrie, E., & Linstead, E. J. (2022). A large-scale sentiment analysis of tweets pertaining to the 2020 US presidential election. Journal of Big Data, 9(1). https://doi.org/10.1186/s40537-022-00633-z