SCALING SYSTEMS SAFELY: ASSURANCE, ETHICS AND GOVERNANCE

Cybernetic or cyber-physical systems (CPS), already surround us, and are increasingly being designed to make decisions that previously required human insight and input. In order to ensure humanity maximally benefits from these new advances in technology, the world will need practitioners who can holistically consider and manage the potential impacts of CPS on humans, each other, and our world. Assurance is the mechanism by which risk is identified and managed in an informed and strategic way. Identifying, predicting, and mitigating risk is already known to be a challenging proposition in complex technical systems, but the possibility that a CPS is capable of evolving beyond its initial state adds an additional dimension to this challenge. There has been a great deal of focus on ethics as it relates to assuring components of CPS, including advanced artificial intelligence, data analytics, algorithmic-based decision making and next generation process automation. If we can program a CPS with ethical guidelines, the reasoning goes, then it will make decisions in line with human interest, even as it learns from its surroundings. But ethics are socio-cultural objects, which is to say they are contextual, cultural, contingent and contested. They change over time, and across geographies and cultures. Building ethics into technical systems is a complicated proposition. Safely scaling technical systems will take more than just an awareness of ethics; it will also require awareness of existing, and the creation of new, assurance and risk management practices. In this context, assurance is the mechanism by which risk is identified and managed in an informed and strategic way.