Deploying OpenStack is NOT Difficult

There are too many articles around at the moment claiming that OpenStack is difficult to set up, and too many vendors claiming the only answer is consulting.

Yes, consulting can be important to get the business side of your private cloud worked out, but setting up OpenStack doesn't need to be difficult, when you have a distribution that's main purpose is ease-of-deployment in enterprise environments.

Here are some factoids about SUSE OpenStack Cloud:

• It was the first enterprise OpenStack distro
• SUSE introduced the concept of "distro" for OpenStack
• It is actually a distro that can be set up by normal people - not just an invitation to a consulting engagement by a vendor
• It is the only distro to ever win Intel's "Rule the Stack" competition for ease-of-installation-and- management (3 times in a row, and sometimes when no-one else was able to complete the task)
• It is the only distro that supports KVM and Xen and VMware and HyperV and Docker and zVM – yes! you can even send workloads to mainframes!
• The deployment tool can deploy SUSE Linux Enterprise and HyperV/Windows Server as compute nodes.
• Installation of SUSE Enterprise Storage (powered by Ceph) is integrated into the deployment tool
• The deployment tool will communicate with your existing infrastructure if you want: plugins make it easy to include your favourite storage system or converged networking
• It is the distro used by some of the well-known OpenStack users
• HA deployment is included with a couple of extra clicks
• SUSE's fleet management tool, SUSE Manager, can be easily integrated into the cloud infrastructure so that new compute/storage/etc nodes automatically get patch/update/security/configuration management
• SUSE's template creation tool, SUSE Studio, can be used to set up the VM's to offer to end users & directly add them into the OpenStack image repository.

Given that ease-of-install has been the hallmark of SUSE OpenStack Cloud since day one, it seems a shame that everyone seems to think it's difficult when it doesn't have to be – it should be a piece of cake.

Some links:

FIS-ASP deploys in one day:  https://www.youtube.com/watch?v=aFdlUIdYwQU

BMW OpenStack + Ceph case study: https://www.susecon.com/doc/2015/sessions/CAS19964.pdf

SAP using SUSE including OpenStack: https://www.youtube.com/watch?v=mLkPzFB1m_w

More (including downloads) at: https://suse.com/cloud

Cloud Adoption in Asia

I was recently asked to compare and contrast the adoption and attitude towards cloud in Asia vs the UK. This is a little less straightfoward as the questioner intended, I think, since “Asia” is a very broad market, with several distinct regions and attitudes to Cloud and Software as a Service.

Still, let's look at how things stand in Japan, Korea, China, Australia, India & South-East Asia.

Japan is still generally a very conservative IT market, however Cloud & SaaS is being looked at carefully. There are definitely examples of major corporations making significant investments in cloud and especially SaaS technologies (for example Toyota is the largest customer of SalesForce.com).   While big enterprise has the resources to move into cloud, and especially private cloud, the chief adopters of public cloud are small-medium enterprise, with adoptions patterns very similar to western countries and for the same reasons of economy.

Korea can be even more conservative than Japan, and generally heavy reliance on older Windows technology (ActiveX, etc) from the early 2000's makes migration to Web 2.x & cloud/SaaS less straightforward in many cases. However, once one large enterprise moves, the rest of the market will follow quickly.

In Australia, cloud technologies are being investigated by large enterprises who have the resources to spare (eg the big banks), and up-coming businesses looking to leapfrog the old fixed-system way of working. Still, the "cloud first" mentality is still a couple of years behind the US.

China shows lots of interest in cloud, with several large service providers offering cloud and SaaS services, and there is a lot of focus on how to develop cloud services with a Chinese flavour rather than straight adoption of western (particular US) systems. In general, government still generally favours traditional application deployment, and security is a significant concern. The adoption trend shows that smaller organisations are most likely to adopt cloud, either by migrating existing process or taking the opportunity to move directly into public cloud. Private cloud are much less common. Larger organisations may see cloud as a way of leapfrogging old deployment methodologies, but the automation and TCO benefits of private cloud can be less attractive where low-cost, high-skilled labour is available. In general it’s still true that there is not a lot of radical new innovation in Chinese IT, but the scale of deployments often pushes the boundaries of technologies.

In South-East Asia, especially outside of Singapore, the scale of IT deployment is generally too small to really gain advantages of on-premise cloud. As with Australia, there are places where small businesses are looking to leapfrog the old way of working & go directly to SaaS, or at least public cloud.  Even this is really more prevalent in Singapore than in the truly emergent economies.

For home-grown enterprises in India  (as opposed to outposts of US, etc companies), cloud & SaaS is an area of  early investigation.  Even more than in China, TCO benefits of private cloud deployments are often undercut by generally low labour costs, which in turn hampers adoption of a lot of operations automation technology.

In comparison to the rest of the world, my general perception is that Asia's cloud-adoption maturity lags behind  that of the US (where adoption of the new can be much faster than elsewhere, even if it's not necessarily a good idea), and also behind the UK and similar western economies.  On the positive side, this means that Asia can take advantage of the mistakes made and lessons learned, and possibly  entirely leapfrog legacy IT concepts that western world has to continue to deal with for many years to come.

From an employment point of view, there is opportunity for people from the US, the UK & the rest of the world to bring their cloud expertise to Asia – most importantly it is business-integration-related skills and experience that are needed, rather than purely technical exposure. The biggest headache most organisations face when moving to cloud is how to integrate business needs with access to the right resources. This is a big change from the way we've been working with IT over the past 30 years (or more) and so needs different approaches & ideas. Experience from people successful in this area will be invaluable to organisations in Asia.

A Turning Point for OpenStack Cloud in the Enterprise

This past couple of days I've been at the CONNECT Expo in Melbourne, which included an OpenStack conference, where I participated as a speaker & panelist.

The focus of that conference was about whether OpenStack is ready for the enterprise, and included contributions from Dave Medbury from Time Warner Cable, Mike Dorman from GoDaddy, and Rik Harris from Telstra, who presented their real-world experiences with OpenStack in commercial settings.

One of the things that was very clear from the conference is that OpenStack is ready for the enterprise. This is probably not news to many who have been following the project for the past few years, but there definitely seems to have been a turning point in recent times, especially enterprise versions of the Juno release (such as SUSE OpenStack Cloud 5) that have become available in the past few months.

The turning point I've really noticed is that OpenStack is becoming much less of a developer's toy or interesting plaything, where new features are the most important aspect of development, and is transitioning (in the core areas at least) to a more practical enterprise-class framework, where component stability, infrastructure high-availability, and robust support options are necessary.  This was reflected in the composition of the audience for the event, which included many more "enterprise architect" types than have been present in the past.  It was also reflected in the questions asked during the panel sessions, which often seemed focused as much on business & organisation as on implementation and technicalities.

So OpenStack is definitely ready for the enterprise, yet as with any complex system and organisational change, there are a few considerations to bear in mind (and many of these are relevant, regardless of the private cloud infrastructure software to be used):

• Implementing cloud computing is an organisational as well as operational change, and should be handled carefully
• Successful cloud implementations rely on an existing IT operations mindset that includes automation & policy-driven deployment (see my previous article).
• Executive-level sponsorship (ideally CIO-level or more) is required to effectively marshall all the disparate IT disciplines towards making cloud deployment a success
• Integration with the lines of business is critical – internal customers of the private cloud have to agree to use the standardised cloud resources, rather than expecting bespoke IT services at cloud prices.
• Not every workload is suitable for cloud deployment right now, as applications really need to be well suited to that kind of environment; this is OK – it's all about using the right tool for the job, so it's best not to try to force the issue and encounter failure.
• The OpenStack control plane must provide High Availability (99.999%+ uptime). Without a highly-available control plane, access to the cloud resources disappears, and even cloud-aware services can fail.
• OpenStack deployment and management can be difficult without the right tools and support, so it makes sense to work with an open source infrastructure software vendor (such as SUSE) to provide the technology, integration, support and training necessary to build up your own capabilities.

There is a growing number of enterprises (including PayPal and BMW) that have adopted OpenStack as a significant part of their IT infrastructure, and this trend will continue as the framework becomes more mature, and as the vendors and other members of the OpenStack Foundation build the body of knowledge in terms of documentation and training.

On that note – SUSE is hiring. At the time of writing there are at least 13 roles open at SUSE related to OpenStack & Cloud. Check https://suse.com/jobs for details.

Bits and Bytes: SUSE® Cloud 4 OpenStack Admin Appliance – An Easier way to Start Your Cloud

Bits and Bytes: SUSE® Cloud 4 OpenStack Admin Appliance – An Easier way to Start Your Cloud: If you used the SUSE Cloud 3 OpenStack Admin Appliance, you know it was a downloadable, OpenStack Havana-based appliance, which even a non-technical user could get off the ground to deploy an OpenStack cloud.

Cloud, high availability, antifragility and so on

In a previous article I wrote about how (the lack of) operational maturity may be impacting the adoption of private cloud in enterprise data centres.  In truth, that's really only half the story: the other significant question is "what applications can be run in the cloud" ?

The majority of "serious" enterprise applications have been around for a long time - think Oracle RDBMS, SAP CRM, and more.  Even if the full client-server or N-tier application stack of these systems has a distributed front-end, at their heart they typically run as monolithic programs that are very tightly integrated into their host computing system and its associated storage and other resources.

A great deal of modern IT design is focused on how to make these systems as resilient as possible, by deploying on robust underlying hardware and software infrastructure, providing redundancy within each host, and providing automated failover and disaster recovery systems to try to ensure there is no single point of failure. When more performance is needed, servers are upgraded with new capacity - hopefully during a seamless internal upgrade of CPU and or memory, but often via a carefully managed and often protracted "lift and shift" migration and update.

The thing is that to a great extent this concept of hardening, protecting, and updating a few known, vital systems runs counter to the "pure" cloud model, which is that cloud-based applications should be independent of the underlying platform & be able to simply scale up or down by adding instances for performance. The application itself should be "antifragile", that is, not need careful maintenance to ensure that it is up and running (this is the "pets vs cattle" analogy).

Antifragility is a term coined by Nassim Taleb (he of  the "Black Swan theory" fame) to describe something that does not merely withstand a shock but actually improves because of it.  Mr Taleb gives a great introduction to Antifragility in his speech at the RSA.  The idea is catching on in the industry: PWC describes how Instagram founders Mike Krieger and Kevin Systrom made use of the concept as they faced the immense problems of scaling their new platform.

The poster-children of cloud computing: Instagram, Netflix, and others, have built their applications (from the ground up) by adopting the antifragile approach. So far, however, traditional software vendors have not yet taken on this methodology, not least because the concepts are too radical for a large portion of their (understandably conservative) customer base.  In this respect we face a chicken-and-egg situation:  without a well-established base of private cloud computing environments to target, software application vendors are unlikely to create products with the cloud in mind.  Simultaneously, without applications that can take advantage of cloud infrastructure, operation managers and systems designers must rely on the traditional approach for making services highly-available, which these days can have the unfortunate side-effect of trying to shoehorn "pets" into an environment that's intended for "cattle"... which in turn yields poor results, frustration, and abandonment of "cloud" as an operational model.

As a "chicken-and-egg" problem there is an obvious solution[1]: start building the private cloud infrastructure for those applications that can make good use of it in the short term: development systems, stateless servers, short-term but frequently needed project infrastructure, and so on. Ideally, data centre managers can re-use their existing infrastructure and virtualisation systems under  Infrastructure-as-a-Service  (IaaS) platform management software, so as not to face a huge and complex migration or the additional expense of a separate silo of equipment just for "cloud".  Meanwhile many enterprise software vendors are working on Software-as-a-Services versions of their own products, in an attempt to capture that particular part of the market. This indicates that when and if cloud computing becomes a well-known operational method for private data centres, the software vendors have already done most of the work to "cloudify" their products.

The short version for IT managers and systems designers:  start building operational experience with private cloud now, and check with vendors about the availability of their products for "real" ("cattle-style") cloud deployment from time to time to assess the viability of moving mission-critical loads to a true cloud environment.



[1] In the case of "chicken-and-egg", the answer is "egg" (from dinosaurs, you see...). 

Why Isn't Cloud Taking Off?

There's been some discussion in the press and from various pundits about how cloud infrastructure projects like OpenStack are "troubled", and how cloud hadn't taken off as expected, especially in the context of private clouds.

First of all, it's worth bearing in mind that in the quest for new news, the hype cycle of much of the media first tends to introduce something with great enthusiasm, make a lot of noise (and possibly overoptimistic predictions) about it, then when it fails to materialise according to the required new news schedule, announces a "failure" or otherwise turns against the original subject (you can see the same thing with reporting of celebrities).

So, cloud is still happening, and is probably inevitable even for the private cloud context at this stage, it's just not running to the timetable of noticeable results that the media would like to see.

But why is this? well it comes down to what "cloud" and especially "private cloud" really is.

At it's most fundamental, cloud computing is simply another way of handling operational management; that is, it's a methodology for organising your IT resources and making them available for use by your organisation.  Cloud computing is attractive because it promises to make more efficient use of resources (up to 80% utilisation, compared to approximately 50% for non-cloud virtualisation, or 30% for physical systems), and also because it promises more nimble access to those resources by project or business units via self-service. 

This efficiency and fast access relies on a number of factors, however. Firstly it relies on automation: without automation in systems configuration & service deployment, addition and access to resources is far too slow, and far to expensive. Cloud also relies on standardised templates for deploying services, even if those templates are as simple as "small", "medium", and "large" [1]. Without templates it's extremely difficult to apply rules to automatically assign workloads to the right systems, and also very hard to do proper pricing & chargeback. Thirdly, cloud relies on effective capacity management, so that new workloads don't suddenly choke the system - especially when end-users are able to self-serve their workloads.

If we look at these general needs of automation, templates, policy-based deployment, capacity planning and so on, we see that there needs to be a reasonably high level of operational maturity in an organisation to be successful at deploying cloud computing.

And this is where many organisations are struggling.

Research firm Gartner has an "operational maturity model" ranging from 0 ("running around with your hair on fire") through to 5 ("IT operations are integrated with the enterprise & provide additional value to the organisation").  The traits and mindset necessary to be truly successful with cloud computing exist around level 4, and perhaps an advanced stage of level 3.  Most IT organisations, for various reasons, usually operate in the region around level 2-3, with very few truly at level 4 and a vanishingly small number at level 5.

This is why, so far, private cloud computing deployments are few and far between: it is difficult stuff, and in many cases although organisations may truly want to adopt cloud operations, there remains a steep learning curve in many of the fundamental concepts that make cloud effective.

Organisations looking to implement cloud should look to develop their existing operational procedures, especially by making use of automation and tools that can handle templates, compliance,and capacity management.

Eventually cloud computing will become the standard method for operation, but there may need to be a generational change in organisational mindset, as well as in software & tools, before it can be fully realised.

Update:  this isn't the only reason, of course. Here's another thought.

1. Proper naming of service templates is important. Another article covers this.

Going For Gold Isn't Enough

For many years, IT services within organisations have been advertised to business projects using a familiar set of names: usually “gold”, “silver” and “bronze”. There may be some noun associated with the service as well, such as “gold processing tier” or “silver storage tier”. The premise behind this kind of naming is that “everyone understands what this means”, and to some extent this is true from a relative point of view – gold is understood to be “better” than silver, which in turn is “better” than bronze. But what does “better” mean in the context of the services on offer? How can someone differentiate between the different services & decide which one to use? What happens when a new service is introduced that fits between the existing ones?

Most of the time the answer to these questions is “no-one knows”, which is less than ideal. The big problem with this precious-metal-related naming methodology (or any other arbitrary method such as colours, gemstones or suchlike), is that although they give a relative indication of “goodness”, they don't actually tell anyone what the service offers, and nor is there any consistency between what a name means from one organisation to the next (or even within a single organisation – they are arbitrary terms, after all). This usually results in a business project simply choosing what sounds like the “best” service (i.e. “gold”), even if the project's needs don't align with what the service is offering. Alternatively a cash-strapped project may choose the “least” option (i.e “bronze”) simply to save on costs, without understanding, for example, that running a database on bottom-tier infrastructure just won't work. This problem is compounded as IT organisations become more like service providers, and especially as automated orchestration is used more to provision and advertise services to business users through self-service portals and the like. It's this context which leads us to understand the way we should be designing and describing our IT services, which is to be as specific as possible.

The concept behind being a service provider (either internally to an organisation or externally), is to consolidate resources and then apportion them to different projects or customers on a per-use basis. The idea is that by offering a standard set of well-defined services, the provider can reduce waste, streamline processes, take advantage of infrastructure efficiencies like data deduplication and single-instance cloning, and save their organisation money (and/or make a margin). The key phrase here is “standard set of well-defined services”: in the service-provider context, customers (or projects) must choose from a menu, rather than being allowed to pick and choose how the ingredients will be combined. In turn, the service provider must be very clear on what is included in the services being offered, and also on what the price will be per unit.

In a NetApp storage context, this means that some understanding of potential workloads must exist in order to develop a service appropriate for each workload the provider wants to cater to, including all of the efficiencies and capabilities relevant to that workload. The goal is to then normalise the per-gigabyte pricing so that simple comparisons can be made on the effective usable space, rather than on the basic physical capacity. For example, likely de-duplication rates for Virtual Server (VSI) workloads in VMware are around 50%, which is greater than the de-duplication rates for simple file sharing at around 20%, so a provider can define different services with per-gigabyte rates for these two workloads that take these savings into account. For example, the “NFS datastore for VMware VSI” might be offered at $0.50/GB, while the “NFS datastore for general data” might be offered at $0.80/GB. This naming and pricing helps the customer or project understand what they are ordering, and helps the service provider direct the customer to the most appropriate service for a workload.

Once you start naming these services in a descriptive way, you can extend them to add additional capabilities. For example the “NFS datastore for VMware VSI” might be extended to a new service called “NFS datastore for VMware VSI with local backup”, which introduces NetApp SnapShots on a defined schedule, and includes an additional per-gigabyte price to cover to cost of capacity used for those backups (perhaps a 20% premium). Another example might be “NFS datastore for general data with Disaster Recovery”, which would include a SnapMirror copy of the data at a remote site, and would have an associated premium to cover both copies of the data, networking costs, and so on.

It's easy to see how infrastructure policies can be applied to basic services to build up a complete catalogue of services, and how the capabilities and efficiencies of the infrastructure can be used to set pricing so that customers or projects can select based on both functional requirements and budget, and service providers can direct customers to the most appropriate infrastructure. With a descriptive naming system, providers can easily introduce new or extended services as technologies become available, without worrying about having to squeeze between or around arbitrary names (is aluminium better or worse than bronze? What sits between silver & gold?). Furthermore, if a service is named according to the functionality it provides, then the underlying technology can be swapped out without necessarily having to re-name the service: for example, if a file sharing service moves from fibre-channel disk to SATA disk plus FlashCache, the customer's view of the functionality remains the same, even though the underlying technologies moved from what might once have been called “gold” storage to what might once have been called “silver”.

So consider making bland old “gold”, “silver” and “bronze” and thing of the past, and well-defined, descriptive services that incorporate infrastructure capabilities the way of the future. It will certainly make life less confusing for your projects & customers.