Things on the edge
Thoughts on technology and business
Previously published July, 20 2015
Part 1 - Command and Control
Most of my professional career has been in spent in Business Intelligence, Master Data Management, and Operational Reporting in heterogeneous IT environments. Of all the IT domains, these areas are driven by two things: data integration and change. Unfortunately, these two issues are also the largest and toughest problems facing an IT organization.
The typical response is to try and centralize these services and enforcing controls to impose such things as canonical data structures, limiting the scope of data, and reducing the number of data providers. This is done to control the rate of change in the environment and to improve visibility. We all know this doesn’t work as effectively as we thought it would when we started. In fact, these controls end up creating critical problems.
It's all about change
The critical problem these controls impose is an inability to change and adapt. If the world is changing and evolving rapidly around you, your natural first reaction will be to try to control chaos. We do this by enforcing rules to impose a model of the reality we want to hold onto. We all know that never works, change is inevitable. The more you try to hold on, the further the real world tends to drift away. Once you see this, it’s too late to recover and someone else now has your customers.
Of the three centralized controls, the first to be implemented will typically be to limit the data providers. The first step will be to limit the software to specific vendors and specific products in each domain. Typically, this will focus on picking the products that have the highest penetration and best vendor relationships. This isn't a bad place to start. Limiting the vendors provides more negotiating leverage and simplified contract management. Limiting the number of applications will also make it easier to support. Taking this further, you could even outsource the entire business process to an outside vendor, eliminating an entire portion of your in house IT department. The entire business case is built on reducing capital and operating expenses. Business shouldn't be run on reducing costs but on delivering value it it's customers. Even in commodity based businesses, there is still room for innovation.
These moves make sense when they are not core business functions or when they don’t provide some competitive advantage. However, people are resistant to change. If you are seeing things from the perspective of the centralized IT organization you’ll see this resistance and call it *Shadow IT.* These are the gorillas out in the field who can’t get with the program, don’t understand what’s at stake, and are being selfish. From their perspective, they are the domain experts with the intimate knowledge of what the business needs to be successful.
Gorilla tactics 101
Agility hinges on the ability to deeply understand the needs of your customers and to be able to respond quickly. Since, I favor the little guy in this fight, let’s look at some of the typical tactics they employ to protect themselves.
Business critically - These systems always evolve from some core painful event. The business asks for a solution and if they are lucky, some smart people get to work right away. If the developers are lucky, a team evolves that builds invaluable domain expertise. It’s a win-win for everyone. The business gets exactly what they need and the developers gain job protection and the satisfaction of making an important contribution to the business.
Business secrecy - Because Central IT would not approve, these new projects are stood up in ways that bypass project governance. They get buried in someone else’s project, or they are kept secret by everyone involved. Famously, these projects also avoid any infrastructure visibility by being run under a desk or never making it out of the lab. If your company happens to have a sophisticated rapid provisioning capability and poor internal billing, you have just removed the largest barrier to this kind of development. Being able to spin up a virtual machine and prototype an application on the internal network makes these deployments incredibly easy and eliminates the need for a developer to maintain their own personal sandbox under their desk. The biggest risk is that, due to the critical nature of the things being developed, the lack of resources, and the need for secrecy, these solutions always cut corners. This technical debt typically weighs a project down to the point where something breaks and the skunkworks project gains a lot of undue attention. Broken business critical functionality will be given all the help that it needs to be fixed. The gig is up.
Technical secrecy - Innocently, that technical debt manifests in code that is difficult to reason out. Unstructured business logic, poor testing, and missing and inadequate documentation become the rule. Also, if the business team isn’t able to protect the development team or keep them engaged with interesting work, the hard earned domain knowledge will be lost as employee churn sets in. Finally, as a result of poor funding, each skunkworks team tends to develop a solution around either their strongest skills or around a set of skills they are willing to learn. These are all innocent mistakes because they naturally evolve and are not an issue when things are running well. Insidiously, all of these things can be done intentionally. Either as a job insurance plan or as a poison pill against the involvement of central IT.