A lot of people interested in technological solutions to democracy’s problems see dry legal texts and think they’re logical enough to automatically translate into code. But being active in computational policy analysis has exposed me to extremely subtle edge cases. After you see them, you can’t unsee how deeply these seemingly objective texts rely on everyday knowledge of our culture. That’s awkward news for computational law.
The hardest examples I’ve found have been while translating lists of criteria into logical statements under the Institutional Grammar framework (Frantz & Siddiki, 2022). There is so much subtlety in disaggregating lists. Real-world policy documents leave their basic logic to context all the time.
Connectives can be left to cultural knowledge
I’ll start with the most dramatic contrast. Compare these two policy sentences detailing hiring qualifications, drawn from examples I’ve actually observed:
- Persons with the following qualifications are eligible to apply:
- five years of experience
- three letters of reference
- up-to-date certifications.
- Persons with the following qualifications are eligible to apply:
- Ph.D.
- J.D.
- M.A.
There is a fundamental logical difference between these statements, and it can only be inferred by cultural knowledge. Do you see it? It’s a big one: While the elements in the first list are linked by AND (all are required), those in the latter are linked by OR (they are alternatives).
There is no explicit cue in the text that these lists are of a completely different nature. In the above cases, the reader must be familiar enough with society to know that the elements in the first are reasonable to expect together (I would disqualify a candidate for having only one), while those in the second would not normally be required for a single position or expected from a single candidate (I would accept a candidate for having only one).
Inclusiveness can be left to cultural knowledge
The items in the second sentence are “disjunctive”: they are linked by OR. An OR can be exclusive or inclusive. That’s the difference between “Select one option” and “Select all that apply.” A college student might be forced to apply their coursework to only the B.A. or B.S. version of a degree (a choice I faced pursuing Cognitive Science; I took the B.A.), but taking more than one course that satisfies that degree’s requirement will not disqualify them (taking both “Philosophy of mind” and “Moral philosophy”). An exclusive “or” (XOR) links the possible choices of degree while an inclusive “or” (OR) links the ways to satisfy that degree.
The link in the second sentence is inclusive because a candidate would not be reasonably disqualified for having earned a Master’s degree on the way to their Ph.D., J.D., or J.D.-Ph.D. But, as often happens in real world policy texts, this inclusivity is left to context. There is nothing explicitly in the rule saying that it’s OK to satisfy more than exactly one of the criteria. You just have to know how degrees work.
Exhaustiveness can be left to cultural knowledge
While inclusivity comes up in both formal and informal uses of “or”, there is another property that only shows up in informal usage. In strictly logical usage, when several statements are linked by AND or OR, at least one must be true in order for the rule to be satisfied; “none” is not an option. But in colloquial usage, a statement can be satisfied even if none of its listed criteria hold. That’s because, in natural language, it is possible for a listing to be “inexhaustive”. If you flag a list of criteria with phrases such as “such as”, “for example”, “including but not limited to”, or “et cetera”, you are creating an inexhaustive listing of a rule’s possible criteria. But real world policy texts aren’t careful enough to utilize these reliably.
The first sentence above is non-exhaustive: an applicant with six years of experience (instead of the stated five) is also probably eligible to apply, even though one who submits four letters of reference (instead of the stated three) could reasonably have their application returned without review. Again, there is no sign in the text, you just have to know enough about society to know that the first criterion is most likely not exhaustive while the second most likely is.
It is possible, and good practice, to flag all of these properties explicitly. For example, “five years of experience” above would be changed to “at least five years of experience”.
Implications
These are really vital and subtle issues in machine-supported reading of policy. They are less of a concern in machine-supported writing. The developer of an authorship support tool must know that their tool should force authors to be explicit about each property, but once that is forced, all outputs will be unambiguous. Computational policy authoring tools like Pika and PolicyKit enforce these distinctions by translating them to the rigorous logic of code (Zhang et al. 2020; Wang et al. 2024). The sentences produced can be explicit by design. The situation is very different in machine reading of human texts that are authored freely, where these ambiguities are endemic.
LLMs
While this problem was once a dealbreaker for natural language processing (NLP), large language models (LLMs) now encode a lot of cultural knowledge as comprehensively as syntactic knowledge. It is now conceivable that an algorithm can determine from context whether items are linked exhaustively, exclusively, and conjunctively. It will still require care, but this is one major way that LLMs represent a major milestone in computational policy analysis and natural language processing generally.
Conclusion
Theses distinctions and caveats aren’t new discoveries. They’re learned by lawyers, policy scholars, linguists, and computer scientists. But I’m sharing anyway because I’m proud of the example text, which very cleanly shows how easy it is to take each of these properties for granted in natural language. Did you notice that the first listing was an AND and the second an OR before I pointed it out? Will you be thinking of them next time you’re writing a list of rules for your community? If so, that’s great. As someone who studies your rules, it would help me out a lot.
References
Frantz, C. K., & Siddiki, S. (2022). Institutional grammar. Springer International Publishing.
Wang, L., Vincent, N., Rukanskaitė, J., & Zhang, A. X. (2024, May). Pika: Empowering Non-Programmers to Author Executable Governance Policies in Online Communities. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (pp. 1-18).
Zhang, A. X., Hugh, G., & Bernstein, M. S. (2020, October). PolicyKit: building governance in online communities. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (pp. 365-378).
