Please email all comments/questions to bms4 [AT] princeton.edu
- Added a news section.
- See a fun demo of stm from Julia Silge.
- Updated supporting packages and papers.
- New version is up on CRAN.
The Structural Topic Model is a general framework for topic modeling with document-level covariate information. The covariates can improve inference and qualitative interpretability and are allowed to affect topical prevalence, topical content or both. The software package implements the estimation algorithms for the model and also includes tools for every stage of a standard workflow from reading in and processing raw text through making publication quality figures.
The package currently includes functionality to:
- ingest and manipulate text data
- estimate Structural Topic Models
- calculate covariate effects on latent topics with uncertainty
- estimate a graph of topic correlations
- compute model diagnostics and summary measures
- create all the plots used in our various papers
- Egami, Fong, Grimmer, Roberts and Stewart. "How to make causal inferences using texts."
- Roberts, Stewart and Nielsen. "Matching Methods for High-Dimensional Data with Applications to Text"
- Roberts, Stewart and Airoldi. "A model of text for experimentation in the social sciences" Journal of the American Statistical Association. 2016.
- Roberts, Stewart and Tingley. Navigating the Local Modes of Big Data: The Case of Topic Models" In Data Analytics in Social Science, Government, and Industry. New York: Cambridge University Press. 2016.
- Lucas, Nielsen, Roberts, Stewart, Storer, and Tingley. "Computer assisted text analysis for comparative politics." Political Analysis. 2015.
- Roberts, Stewart, Tingley, Lucas, Leder-Luis, Gadarian, Albertson, and Rand. "Structural topic models for open-ended survey responses." American Journal of Political Science. 2014.
- Roberts, Stewart, Tingley, and Airoldi. "The Structural Topic Model and Applied Social Science." Advances in Neural Information Processing Systems Workshop on Topic Models: Computation, Application, and Evaluation. 2013.
- Schwemmer. stminsights: A Shiny Application for Inspecting Structural Topic Models. A shiny GUI with beautiful graphics.
- Woloszynek. themetagenomics: Exploring Thematic Structure and Predicted Functionality of 16s rRNA Amplicon Data. . STM for rRNA data.
- Johannesson. "tidystm: Extract (tidy) effects from estimateEffect" . Makes it easy to make ggplot2 graphics for STM.
- Zangri, Tingley, Stewart. "stmgui: Shiny Application for Creating STM Models" . This is a Shiny GUI for running basic STM models.
- Freeman, Chuang, Roberts, Stewart and Tingley. "stmBrowser: An R Package for the Structural Topic Model Browser.'' . This D3 visualization allows users to interactively explore the relationships between topics and the covariates estimated from the stm package in R. See an example here .
- Coppola, Roberts, Stewart and Tingley. "stmCorrViz: A Tool for Structural Topic Model Visualizations." . This package uses `D3 to generate an interactive hierarchical topic explorer.
If you have published a paper using stm that you would like to see included here please email us.
- Nelson, Laura K. "Computational Grounded Theory: A Methodological Framework" Sociological methods & Research 2018.
- Bohr and Dunlap. "Key Topics in environmental sociology, 1990–2014: results from a computational text analysis"Environmental Sociology 2018.
- Banks, Woznyj, Wesslen and Ross. "A Review of Best Practice Recommendations for Text Analysis in R (and a User-Friendly App)"Journal of Business and Psychology 2018.
- Hagen, Harrison and Dumas. "Data Analytics for Policy Informatics: The Case of E-Petitioning" Policy Analytics, Modelling, and Informatics 2018.
- Kuhn, Kenneth D. "Using structural topic modeling to identify latent topics and trends in aviation incident reports" Transportation Research Part C: Emerging Technologies 2018.
- Tvinnereim, Flottum, Gjerstad, Johannesson and Nordo. "Citizens’ preferences for tackling climate change. Quantitative and qualitative analyses of their freely formulated solutions "Global Environmental Change 2017.
- Mildenberger and Tingley. "Beliefs about Climate Beliefs: The Importance of Second-Order Opinions for Climate Politics"British Journal of Political Science 2017.
- Terman. "Islamophobia and Media Portrayals of Muslim Women: A Computational Text Analysis of US News Coverage "International Studies Quarterly 2017.
- Chakrabarti and Frye. "A mixed-methods framework for analyzing text data: Integrating computational techniques with qualitative methods in demography"Demographic Research 2017.
- Bail, Brown and Mann. "Channeling Hearts and Minds: Advocacy Organizations, Cognitive-Emotional Currents, and Public Conversation "American Sociological Review 2017.
- McInerney, Doherty, Bindoff, Robinson and Vickers. "How is palliative care understood in the context of dementia? Results from a massive open online course"Palliative Medicine 2017.
- Gupta, Wang, Lin, Hong, Sun, Liebman, Stern, Dasgupta and Roberts. "Toward Building a Legal Knowledge-Base of Chinese Judicial Documents for Large-Scale Analytics"Legal Knowledge and Information Systems 2017.
- Zhang, Qiang and Jiang. "Finding Academic Concerns on Real Estate of U.S. and China: A Topic Modeling Based Exploration"Proceedings of the 21st International Symposium on Advancement of Construction Management and Real Estate 2017.
- Chow, Kumar, Ouyang, Zhong, Lee and Inverso. "What can Physicians learn from Social Forums: Insights from an on-line Self Help and Support Group" Computational Advances in Bio and Medical Sciences (ICCABS), 2017 IEEE 7th International Conference 2017.
- Moeller, Munksgaard and Demant. "Flow My FE the Vendor Said: Exploring Violent and Fraudulent Resource Exchanges on Cryptomarkets for Illicit Drugs"American Behavioral Scientist 2017.
- Gwak and Sohn. "Identifying the trends in wound-healing patents for successful investment strategies "PLOS One 2017.
- Light, Ryan and Colin Odden. "Managing the Boundaries of Taste: Culture, Valuation, and Computational Social Science" Social Forces 2017.
- Kuhn, Kenneth D. "Topics and Trends in Incident Reports: Using Structural Topic Modeling to Explore Aviation Safety Reporting System Data" Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017) 2017: 1-10.
- Kim, In Song. "Political Cleavages within Industry: Firm-level Lobbying for Trade Liberalization" American Political Science Review 2017.
- Tingley, Dustin. "Rising Power on the Mind." International Organization. 2017.
- Tvinnereim, Endre, Xiaozi Liu, and Eric M. Jamelske. "Public perceptions of air pollution and climate change: different manifestations, similar causes, and concerns." Climatic Change 2016: 1-14.
- Truex, Rory. Making Autocracy Work. Cambridge University Press. 2016.
- Kolar, Mladen and Matt Taddy. "Discussion of 'Coauthorship and Citation Networks for Statisticians'" The Annals of Applied Statistics 2016.
- Bauer, Paul C., Pablo Barberá, Kathrin Ackermann, Aaron Venetz. "Is the Left-Right Scale a Valid Measure of Ideology? Individual-Level Variation in Associations with "Left"" and "Right"" and Left-Right Self-Placement" Political Behavior 2016.
- Sachdeva, Sonya, Sarah McCaffrey and Dexter Locke. "Social media approaches to modeling wildfire smoke dispersion: spatiotemporal and social scientific investigations." Information, Communication & Society. 2016.
- Munksgaard, Rasmus and Jakob Demant. "Mixing politics and crime- thre prevalence and decline of political discourse on the cryptomarket." International Journal of Drug Policy. 2016.
- Huff, Connor and Dominika Kruszewska. "Banners, Barricades, and Bombs The Tactical Choices of Social Movements and Public Opinion" Comparative Political Studies. 2016.
- Bail, Christopher A. "Cultural carrying capacity: Organ donation advocacy, discursive framing, and social media engagement." Social Science & Medicine. 2016.
- Law, David S. "Constitutional Archetypes" Texas Law Review. 2016.
- Farrell, Justin. "Corporate funding and ideological polarization about climate change" Proceedings of the National Academy of Sciences. 2016.
- Wang, Baiyang and Diego Klabjan. "Temporal Topic Analysis with Endogenous and Exogenous Processes." Thirtieth AAAI Conference on Artificial Intelligence. 2016.
- Reich, Stewart, Mavon and Tingley "The Civic Mission of MOOCs: Measuring Engagement across Political Differences in Forums." Association for Computing Machinery: Learning at Scale. 2016.
- Tvinnereim, Endre and Kjersti Flottum. "Explaining topic prevalence in answers to open-ended survey questions about climate change" Nature Climate Change. 2015.
- Mishler, Alan, Erin Smith Crabbm Susannah Paletz, Brook Hefright, Ewa Golonka. "Using Structural Topic Modeling to Detect Events and Cluster Twitter Users in the Ukrainian Crisis." International Conference on Human-Computer Interaction. 2015.
- Milner, Helen and Dustin Tingley. Sailing the Water's Edge: The Domestic Politics of American Foreign Policy . Princeton University Press. 2015.
- Romney, David, Brandon Stewart and Dustin Tingley. " Plain Text: Transparency in the Acquisition, Analysis, and Access Stages of the Computer-assisted Analysis of Texts." Qualitative and Multi-Method Research. 2015.
- Genovese, Federica. "Politics ex cathedra: Religious authority and the Pope in modern international relations" Research & Politics 2015.
- Reich, Tingley, Leder-Luis, Roberts and Stewart. "Computer-Assisted Reading and Discovery for Student Generated Text in Massive Open Online Courses" Journal of Learning Analytics. 2015.
The package is available on CRAN and can be installed using:
You can always get the most stable development release from the Github repository. Assuming you already have R installed (if not see http://www.r-project.org/), the easiest way to install from the Github repository is to use the devtools package. First you have to install devtools using the following code. Note that you only have to do this once
Then you can load the package and use the function
Note that this will install all the packages suggested and required to run our package. It may take a few minutes the first time, but this only needs to be done on the first use. In the future you can update to the most recent development version using the same code.
See the vignette for several example analyses. The main function to estimate the model is
stm() but there are a host of other useful functions. If you have your documents already converted to term-document matrices you can ingest them using
readCorpus(). If you just have raw texts you will want to start with
Have a large text corpus or need a language we don't provide support for? See our sister project txtorg