If you have Shadow IT in your world, you have already failed. - Part 2

Part 2 - A culture of innovation
Previously published July, 22 2015
​
See Part 1
The culture of the organization determines how the Shadow IT evolves and whether or not it is a problem. If the culture has a more centralized structure but also supports innovation, autonomy, and is customer focused, Shadow IT becomes a Skunkworks that can be managed through governance, openness, and creating ways for teams to share their insights and experiences. This can take many flavors and has become the cornerstone of many companies. The most famous examples are Amazon and Netflix, who have adopted the policy of “If you build it, you run it”. Teams are given wide latitude to innovate and the results speak for themselves.

This DevOps mantra is really a way govern the masses. Encouraging technical openness by open sourcing code and making that code easily visible using tools like GIT can dramatically improve quality and transparency. True participatory democracy in action.

In return, Central IT becomes a caretaker with two huge responsibilities. First, they need to facilitate communication between teams because secrecy is diametrically opposed to communication and sharing. The role of IT is to break down these barriers and get the smart people talking. They will start sharing and realigning naturally. Once that happens, real business value will be created that would have been impossible before. 

I think you could also make a strong case that low level governance isn’t necessary because smart customer focused people will govern themselves. In this situation, governance revolves around identifying the non-business critical activities thereby letting teams focus on what's important. Governance will evolve out of what the customers need bounded by "Big Rules" of culture and technology. 

IaaS - Infrastructure as a service
The role of IT now shifts to providing the tools to allow teams to dramatically improve their agility and collaboration. It isn't enough to make the transition from dedicated servers to "the cloud". IT must truly embrace their role as facilitators to the business and to development teams. There needs to be a holistic view of what Infrastructure as a Service really means. The key word isn't Infrastructure, it is Service. Service in the sense of providing everything a customer needs better than anyone else. So what do they need?

You must be this tall
Martin Fowler, in his post Microservice Prerequisites sums up the bare minimum capabilities of an IaaS environment as the following:

Rapid provisioning: You should be able to fire up a new server in a matter of hours (or less). Naturally this fits in with Cloud Computing, but it's also something that can be done without a full cloud service. To be able to do such rapid provisioning, you'll need a lot of automation - it may not have to be fully automated to start with, but to do serious microservices later it will need to get that way.

Basic Monitoring: With many loosely-coupled services collaborating in production, things are bound to go wrong in ways that are difficult to detect in test environments. As a result, it's essential that a monitoring regime is in place to detect serious problems quickly. The baseline here is detecting technical issues (counting errors, service availability, etc) but it's also worth monitoring business issues (such as detecting a drop in orders). If a sudden problem appears then you need to ensure you can quickly rollback, hence…

Rapid application deployment: With many services to manage, you need to be able to quickly deploy them, both to test environments and to production. Usually this will involve a Deployment Pipeline that can execute in no more than a couple of hours. Some manual intervention is all right in the early stages, but you'll be looking to fully automate it soon.

These sound like technologies but they are really about culture.

These capabilities imply an important organizational shift - close collaboration between developers and operations: the DevOps culture. This collaboration is needed to ensure that provisioning and deployment can be done rapidly, it's also important to ensure you can react quickly when your monitoring indicates a problem. In particular any incident management needs to involve the development team and operations, both in fixing the immediate problem and the root-cause analysis to ensure the underlying problems are fixed.

[...] If you don't have these capabilities now, you should ensure you develop them so they are ready by the time you put a microservice system into production. Indeed these are capabilities that you really ought to have for monolithic systems too. While they aren't universally present across software organizations, there are very few places where they shouldn't be a high priority. - Martin Fowler