Measuring Institutional Variation Across American Indian Constitutions Using Automated Content Analysis

How can variation in institutions over time and across jurisdictions be measured effectively? This is an issue of great importance to scholars studying how institutional characteristics shape political, social and economic issues such as development and the rule of law. We present a new dataset of American Indian and Alaska Native (AIAN) constitutions and a new approach to measure variation in polities across AIAN nations. Studying AIAN polities can provide insights into how institutions – in AIAN nations and more generally – shape local policy outcomes. Existing data on AIAN institutions have largely been based on costly and time-consuming expert coding and survey approaches, where the end product will become obsolete once institutions change. We implement a text-as-data approach using machine learning techniques to compare institutional designs across AIAN nations, studying in particular the variation of judicial institutions, which have previously been shown to play a crucial role in AIAN development. We present results for a sample of 97 American Indian constitutions and compare our machine coded measures against previous attempts to systematically code AIAN institutions. We show that machine coding replicates expert coded data. Our automated content analysis of the full texts of constitutional documents allows for more flexible and customizable measurement of the variation of institutions in AIAN communities, while using a larger corpus of data than existing approaches limited by costs related to data collection and coding. Our approach can also easily be extended to compare other topics, such as the executive, and shows the potential of automated measures to complement or confirm more traditional coding of political institutions.