By Joanna Wang
Editor's Note: This is part 2 of a mini-series on the 2025 EAAMO Conference.
Part 1 | Part 3
During EAAMO 2025, I attended a summit on Nov 7th which separated participants into two groups, vendors and procurers, to create a framework for principles surrounding AI technology sales and implementations using local government as a context. As a previous AI application researcher, I joined the vendor group. Through my participation in this group, I noticed that significant gaps exist not only between vendors and procurers at large, but specifically between salesman and developers. As such, we came up with a set of principles tailored for the two parties we cater to, the government (procurers) and citizens (beneficiaries), organized into principles which maximized benefits and principles which minimized harm for each of these parties.
Maximizing Benefits
Minimize Cost
Vendors tend to minimize the costs in both money and time in sales contexts. However, salesmen may have a disconnect with engineers, leading to underestimations. For example, salesmen may not expect the uncertainty in training models. Keeping costs low but realistic will make the lives of procurers easier.
Justify Need
Procurers should have deep understanding of the problem they are solving. As helpers, vendors need to detail use cases and provide metrics to demonstrate the effectiveness of product they have developed.
Enhance Agency
AI model are often considered to be "blackboxes." As such, it can be difficult to provide procurers with the tools needed to get the most out of AI-based systems. Consider using tools such as XAI or introducing a human into the loop to try to provide some clarity, but be careful: tools like XAI are slower and more expensive, and putting humans into the loop can introduce bias.
Low Burden
AI systems should interrupt the public's day-to-day life as minimally as possible. While it's important to be transparent surrounding use of AI systems in communities, keeping these tools out of the way will allow them to improve lives invisibly.
Minimizing Harm
Research Vendors
The reputation of a vendor matters to both the government and its citizens. Vendors have vested interests in selling their products, so internal metrics readily presented should be met with healthy skepticism. Some vendor metrics procurers may want to consider asking for or researching themselves include:
- Number of lawsuits faced
- Performance reviews on past projects
- Political motivations
- Workforce size
- Training resources for procurers
Ensure Auditability
As previously mentioned, AI systems can be difficult to pick apart due to their black box nature. However, procurers should still be judicious about the things vendors can be transparent about, such as frequency of documentation updates, technical and quarterly reports, contracting rights, and any past delays on any of those documents.
Protect Privacy
Citizens highly value their privacy. Recognizing this, vendors should be able to explicitly describe if and how the data their systems will collect from the public is anonymized, as well as how long it is kept and who will have access to it.
Evaluate Bias
Many AI systems are known to face issues with biases. Vendors should provide metrics quantifying any biases and describing steps taken to correct it, but procurers should also play an active role in this process, evaluating potential biases of vendors as a whole and considering how those may play into any models contracted.
Through my work at this summit, I had a unique opportunity to think from a procurer's perspective. As a vendor, I understand their worries surrounding the process, from selecting vendors to checks and audits. Despite our group's expertise, though, these principles will still require future discussion, as the gap between procurers and vendors they reflect is greater than ever. The reality is that both auditions and implementations of AI systems require procurers and citizens, not just vendors, to have basic knowledge about AI, which is why it’s so important to promote basic education on AI knowledge within our local communities.
If you're interested in learning more, you can read our slides fully explaining these principles.