Please email all comments/questions to bms4 [AT] princeton.edu
The Structural Topic Model is a general framework for topic modeling with document-level covariate information. The covariates can improve inference and qualitative interpretability and are allowed to affect topical prevalence, topical content or both. The software package implements the estimation algorithms for the model and also includes tools for every stage of a standard workflow from reading in and processing raw text through making publication quality figures.
The package currently includes functionality to:
- ingest and manipulate text data
- estimate Structural Topic Models
- calculate covariate effects on latent topics with uncertainty
- estimate a graph of topic correlations
- compute model diagnostics and summary measures
- create all the plots used in our various papers
- Roberts, Stewart and Airoldi. "A model of text for experimentation in the social sciences" Journal of the American Statistical Association. 2016.
- Roberts, Stewart and Nielsen. "Matching Methods for High-Dimensional Data with Applications to Text"
- Roberts, Stewart and Tingley. Navigating the Local Modes of Big Data: The Case of Topic Models" In Data Analytics in Social Science, Government, and Industry. New York: Cambridge University Press. 2016.
- Roberts, Stewart, Tingley, and Airoldi. "The Structural Topic Model and Applied Social Science." Advances in Neural Information Processing Systems Workshop on Topic Models: Computation, Application, and Evaluation. 2013.
- Roberts, Stewart, Tingley, Lucas, Leder-Luis, Gadarian, Albertson, and Rand. "Structural topic models for open-ended survey responses." American Journal of Political Science. 2014.
- Lucas, Nielsen, Roberts, Stewart, Storer, and Tingley. "Computer assisted text analysis for comparative politics." Political Analysis. 2015.
- Roberts, Stewart and Tingley. "stm: R Package for Structural Topic Models"
- Zangri, Tingley, Stewart. "stmgui: Shiny Application for Creating STM Models" . This is a Shiny GUI for running basic STM models.
- Freeman, Chuang, Roberts, Stewart and Tingley. "stmBrowser: An R Package for the Structural Topic Model Browser.'' . This D3 visualization allows users to interactively explore the relationships between topics and the covariates estimated from the stm package in R. See an example here .
- Coppola, Roberts, Stewart and Tingley. "stmCorrViz: A Tool for Structural Topic Model Visualizations." . This package uses `D3 to generate an interactive hierarchical topic explorer.
If you have published a paper using stm that you would like to see included here please email us.
- Light, Ryan and Colin Odden. "Managing the Boundaries of Taste: Culture, Valuation, and Computational Social Science" Social Forces 2017.
- Kuhn, Kenneth D. "Topics and Trends in Incident Reports: Using Structural Topic Modeling to Explore Aviation Safety Reporting System Data" Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017) 2017: 1-10.
- Kim, In Song. "Political Cleavages within Industry: Firm-level Lobbying for Trade Liberalization" American Political Science Review 2017.
- Tingley, Dustin. "Rising Power on the Mind." International Organization. 2017.
- Tvinnereim, Endre, Xiaozi Liu, and Eric M. Jamelske. "Public perceptions of air pollution and climate change: different manifestations, similar causes, and concerns." Climatic Change 2016: 1-14.
- Truex, Rory. Making Autocracy Work. Cambridge University Press. 2016.
- Kolar, Mladen and Matt Taddy. "Discussion of 'Coauthorship and Citation Networks for Statisticians'" The Annals of Applied Statistics 2016.
- Bauer, Paul C., Pablo Barberá, Kathrin Ackermann, Aaron Venetz. "Is the Left-Right Scale a Valid Measure of Ideology? Individual-Level Variation in Associations with "Left"" and "Right"" and Left-Right Self-Placement" Political Behavior 2016.
- Sachdeva, Sonya, Sarah McCaffrey and Dexter Locke. "Social media approaches to modeling wildfire smoke dispersion: spatiotemporal and social scientific investigations." Information, Communication & Society. 2016.
- Munksgaard, Rasmus and Jakob Demant. "Mixing politics and crime- thre prevalence and decline of political discourse on the cryptomarket." International Journal of Drug Policy. 2016.
- Huff, Connor and Dominika Kruszewska. "Banners, Barricades, and Bombs The Tactical Choices of Social Movements and Public Opinion" Comparative Political Studies. 2016.
- Bail, Christopher A. "Cultural carrying capacity: Organ donation advocacy, discursive framing, and social media engagement." Social Science & Medicine. 2016.
- Law, David S. "Constitutional Archetypes" Texas Law Review. 2016.
- Farrell, Justin. "Corporate funding and ideological polarization about climate change" Proceedings of the National Academy of Sciences. 2016.
- Wang, Baiyang and Diego Klabjan. "Temporal Topic Analysis with Endogenous and Exogenous Processes." Thirtieth AAAI Conference on Artificial Intelligence. 2016.
- Reich, Stewart, Mavon and Tingley "The Civic Mission of MOOCs: Measuring Engagement across Political Differences in Forums." Association for Computing Machinery: Learning at Scale. 2016.
- Tvinnereim, Endre and Kjersti Flottum. "Explaining topic prevalence in answers to open-ended survey questions about climate change" Nature Climate Change. 2015.
- Mishler, Alan, Erin Smith Crabbm Susannah Paletz, Brook Hefright, Ewa Golonka. "Using Structural Topic Modeling to Detect Events and Cluster Twitter Users in the Ukrainian Crisis." International Conference on Human-Computer Interaction. 2015.
- Milner, Helen and Dustin Tingley. Sailing the Water's Edge: The Domestic Politics of American Foreign Policy . Princeton University Press. 2015.
- Romney, David, Brandon Stewart and Dustin Tingley. " Plain Text: Transparency in the Acquisition, Analysis, and Access Stages of the Computer-assisted Analysis of Texts." Qualitative and Multi-Method Research. 2015.
- Genovese, Federica. "Politics ex cathedra: Religious authority and the Pope in modern international relations" Research & Politics 2015.
- Reich, Tingley, Leder-Luis, Roberts and Stewart. "Computer-Assisted Reading and Discovery for Student Generated Text in Massive Open Online Courses" Journal of Learning Analytics. 2015.
The package is available on CRAN and can be installed using:
You can always get the most stable development release from the Github repository. Assuming you already have R installed (if not see http://www.r-project.org/), the easiest way to install from the Github repository is to use the devtools package. First you have to install devtools using the following code. Note that you only have to do this once
Then you can load the package and use the function
Note that this will install all the packages suggested and required to run our package. It may take a few minutes the first time, but this only needs to be done on the first use. In the future you can update to the most recent development version using the same code.
You can also grab the binaries or source files for the latest release here: (https://github.com/bstewart/stm/releases). Then use
repos=NULL so that
install.packages(filepath, repos = NULL)
See the vignette for several example analyses. The main function to estimate the model is
stm() but there are a host of other useful functions. If you have your documents already converted to term-document matrices you can ingest them using
readCorpus(). If you just have raw texts you will want to start with
Have a large text corpus or need a language we don't provide support for? See our sister project txtorg