Abstraction and the management of the multicloud complexities

6 min readDec 2, 2020

While Enterprises continue to focus their investments on a single public cloud provider, there is an increasing number of organizations using two or more cloud providers. The benefits of a multicloud strategy are covered in many articles; for example, avoid vendor lock-in increase agility, workload optimization, benefit from cloud providers’ unique capabilities (e.g. offerings in AI, IoT, ML etc.).

However, multicloud adds significant complexities and implementing and managing a multicloud environment has many challenges. These can prevent the organisation from realising the benefits of their public cloud platform. Thus, it is important for Enterprises who would like to adopt a multicloud strategy to have a plan on how to manage these complications to realise the anticipated ROI.

Understand the business goals and requirements

Before jumping to technology and architecting a multicloud (or a single cloud) platform, it is important to understand the problem the organisation is trying to solve and the current state of the applications and workloads.

There are different areas where the organisation should gather requirements in order to manage the complexities better and overcome any challenges that may arise:

The owners’ and the end-users’ requirements for each application. It is important to reach out to all stakeholders of each application and capture their requirements (functional and non-functional).
The performance and SLA requirements for each application. At this stage, it is important to remember that performance is impacted by several components of the platform (e.g. network, databases, storage, APIs, CPU) and it is measured by the lowest component in the performance chain.
The application and development requirements, like programming languages, databases, operating systems etc.
The security requirements for each application. This includes encryption requirements, IAM, any security regulation around storage of data (e.g. PII, patient data) etc.

These areas are applicable regardless of the organisation’s plans to use one or more cloud providers; however, in a multi-cloud strategy, the complexity of capturing and managing these requirements increases significantly.

The business goals, the current state and requirements will help the organisation define their (multi)cloud migration plan. The organisation must have a holistic migration plan which includes the appropriate cloud provider and services, workload priorities, milestones and timelines etc. However, if the organisation’s strategy is multicloud, it is important to include approaches and frameworks to manage the complexities which come with a multicloud platform.

Manage complexity through abstraction

An approach to minimise the multicloud complexities is to take a holistic view of the organisation’s platform and create layers of abstraction across the various domains of the platform (e.g. security, governance, operations, data and storage etc.). With this approach, the organisation must review the requirements of each domain and start abstracting the processes, tools and other solutions to minimise the complexities of the underlying cloud components and services.

Storage and data

For example, when the teams look at the storage and data domain, they need to gather requirements holistically around:

Capacity and growth requirements. It is crucial to not only focus on the current capacity needs but take into consideration the storage demand in the future and understand how these data are going to be used.
Speed (response time, IOPS etc.). As mentioned previously, the performance might be related to multiple components of the platform (compute, network etc.), and these will be related to data performance.
Storage models, e.g. block, object, file, database. It is important to look for better ways to organise the data across the environment and remember that the data serve business needs.
Backup and disaster recovery requirements
Security across data in transit and at rest, e.g. IAM, encryption etc.

Once, the data and storage requirements are well understood, we can abstract the processes and tools to manage and access the physical data (e.g. databases, object storage etc.). For example, depending on the requirements, they teams might decide to create specific versions of the data which exist in memory, establish a shared process for data hygiene and define a single source of truth for the various datasets. The abstracted processes and tools must span across the multicloud environment to provide a unified layer for accessing and managing the underlying physical data.

Security and compliance

Similarly, when the teams look at security and compliance, they need to ensure that these two aspects of the platform are not an afterthought. There are several areas the organisations need to understand the requirements. For example:

Policies across the organisation and guardrails across the various services and resources (e.g. limits in storage usage, compute types and times etc.)
Laws and regulations compliance requirements.
IAM and encryption requirements (at rest and in transit). For example, they must avoid binary access to services and resources (e.g. all or nothing roles) and define the appropriate hierarchy which will allow the organisation to allow or block access to specific areas of the multicloud platform.
Current directory integration (including service, people and resources directories) and integration requirements in the multicloud environment.

When the governance and security requirements are captured, and the future multicloud requirements are understood, the company can start abstracting the policies and tools and identify patterns. It is important to ensure that the defined policies can be shared and that governance and security are linked. For example, policies for resources and services should interact with security, which is linked with application and data. The tools will also need to integrate with other domains and allow us to invoke policies in a specific order. Automation is also essential and will allow the organisations to manage the complexities as the environment grows.

Cloud operations

The ability to monitor and operate the solutions across the various domains in the long term is where the organisations understand the value. Thus, CloudOps is very important and some areas the organisations need to have good visibility are:

How the teams deal with configuration changes and issue fixes currently and future requirements in a multicloud environment (e.g. self-healing systems)
Performance management and monitoring requirements (e.g. real-time, near real-time etc.). It is crucial to keep in mind that the applications must also be monitored, not only the infrastructure. In addition, it is important to understand the dashboards and data requirements for monitoring and alerting and the cost associated.

In order to manage the complexities of managing and operating in a multicloud environment, the organisation must abstract the processes and tools and define patterns based on best practices. Automation, including automated diagnostics and repair, is crucial as it will allow sequencing actions based on events and allow operations to scale as the environment grows. A level of abstraction will allow the organisation to invoke processes to manage or access the underlying physical or virtual services.

Wrap up

Multicloud can have several benefits and refining things, becoming more efficient and reducing cost in the cloud is an on-going journey. Enterprises can leverage abstraction approaches and automation to manage and reduce complexities in a multicloud environment using existing resources. There are other domains which the Enterprises will need to look for reducing complexities when they move to a multicloud environment (e.g. services domain, cognitive domain). The course “Cloud Complexity Management for Multicloud Deployments” offers a great source of valuable information across all the various domains.