When I embarked on this project, I underestimated how difficult it would be. Learning the program and putting the documents into the necessary format took way more time than I originally expected. As a result, I had to limit the size of my corpus. Instead of looking at an entire decade of foreign policy documents, I focused on the years from 1951 to 1954. One of the benefits of having such a small corpus was that I was intimately familiar with the historical context of the corpus and many of the documents themselves. After generating the results, I was pleased to discover that the model generated many of the topics I expected to see. Rather than being disappointed because it didn’t reveal any new avenues for research, it confirmed for me the efficacy of topic modeling in general. I have a greater desire now that I know the model works to apply it to a much larger corpus.
Despite the small size of my corpus, there were occasions when the results prompted me to investigate something further. A combination of topic modeling and close reading allowed me to explore new avenues. For example, the presence of “tudeh” and the absence of religion prompted me to ask different questions and question current scholarship. While these new directions confirmed, more than challenged, an already held view, it still demonstrates how a topic model can get you to think about a subject in a new way.
This project also gave me the opportunity to make some mistakes and learn from them for future research. I didn’t realize the importance of labeling my documents with the date and document number so that I could map my findings over time. This would have been a really interesting addition to my research. Other scholars who have done such projects, such as Cameron Blevin, David Allen, and Matthew Connelly, discovered some fascinating trends that would not have been obvious through a close reading.
Overall, I would highly recommend people give this program a try. It’s a lot of work, but it really does allow you to see patterns that would otherwise have remained hidden.
Attached you will find my final paper. Enjoy!