Designing Data-oriented Processes
An important part of my job – which I don’t recall ever learning in school – is designing data-oriented processes. I can program in a number of computer programs. Writing a computer program certainly involves designing a process. Many people have the experience of being at a line-up – perhaps at an airport or cashier. At some point, somebody designed that line premised on how people would be processed. A web designer might create an online registration system for students that wish to enroll in courses. So it isn’t unusual for people to construct elaborate processes. The main difference as it relates to my job is that my processes create persistent assets – such as databases – normally intended to enable analysis for business purposes.
I want to emphasize that there are actually two basic operations taking place in data-oriented processes: 1) the underlying activity such as signing up new customers; and 2) the data-system that is connected to the activity. I am routinely in situations where the process does not already exist – and so I must create it from scratch. At the end, both the process and the data system must be in place. By using the term “system,” I know that some readers might be thinking of a computer application. If only life were that simple. The system is how the activity gives rise to data – and of course how the data gives rise to insights.
Most of the systems that I use today didn’t exist before I designed and made use of them. Recently, I was asked to handle our performance incentive program. Since the program never existed before, there were no established logistics. I had to rely my skills and experiences to build the system. Much the same can be said of our quality control system, which was once paper-based, fairly rudimentary, and didn’t leave much of a data trail at all. These days, apart from the program and its underlying activities, I always look at the resulting data trail as a resource or asset. In my work, I create assets. It is an asset for me because I can provide analysis and advice using the data. It is also an asset for the organization, which can better control operations.
Apart from accomplishing the original intended activity, data-oriented processes open many opportunities. Yet I find that many data scientists focus on the microscopic – the analysis of data – as if a computer can’t do this work far more efficiently than humans. They ignore the macroscopic – the underlying processes; the connection to the data; the linkages between the data, employees, and managers; and the integral bonds between all of this and the objectives of the organization. So no, there isn’t a computer on earth that can do this part of my job. And no – no machine can replace me.
I am not aware of any school programs that teach students how to deal with these holistic challenges. There are certainly administration and management programs. These tend to be disassociated from the practical day-to-day, hands-on, and face-to-face needs of companies. More importantly, there isn’t much discussion on how to create data-oriented systems from common organizational behaviours. Speaking for myself, given my access to enormous amounts of actionable data through years of diligent use of systems that I designed, nothing is more inspiring than systems dynamically aligned with operational needs, getting things done while opening up more opportunities for management.
Link: Designing Data-oriented Processes