I took notes during Rod Johnson’s keynote at the S2G Forum in London this morning, which was entitled “What can the cloud do for me?” In the interests of timely publishing, I present the notes pretty much as-is here.

Introduction:

Last year everyone was saying , “the shift to the cloud *is* going to happen.” The question was how can Java developers contribute and reap the benefits? The Java community can be sceptical of new developments. There has been less of a bottom-up movement in cloud for Java than in other language communities (e.g. Java). We want Java to continue to be a vital platform.

Industry trends:

New client platforms: smartphones overtake PCs in Q4 2010. When we moved from e.g. Entity beans to Spring + Hibernate there was no fundamental change in what apps were trying to do (just in how they achieved those ends). In the last 3 years it is suddenly

…not just about implementing apps in different ways, it is about implementing different requirements.

Smartphones and tablets will form a very clear majority of internet devices. We are also seeing the rise of the native application (whereas until relatively recently we were all assuming the browser would be the universal client). 350K+ apps available for download in app stores, 10 billion total downloads from app stores in 2010, 100 billion projected app downloads from stores in 2013.

Data explosion: interaction patterns from e.g. a smartphone are different. Connectivity may be intermittent, expect caching on the device. There are more fine-grained interactions vs. the less frequent coarser interactions we used to see. Data volumes are increasing at 60% per year. Traditional RDBMS scaling is vertical, not horizontal, and this causes issues with elastic scale. Furthermore, ACID semantics are not needed for all use cases. As a manager at VMware, Rod uses 4 different SaaS apps that run outside of our data centers. Therefore when VMware’s IT department writes internal apps, they have characteristics of mashups! ACID concerns don’t apply in same way. Distributed applications need *distributed data*.

It is abundantly clear that not all of this new data can go in a relational db. 

Rod encouraged the audience to take an interest in new data storage options. The change in the data landscape is one of the biggest single things affecting developers regardless of programming language. It is worth investing time to learn about the new models and use cases. Examples: key-value stores, column stores, document stores, graph stores. One of the challenges for all of these stores is that they are relatively hard for developers to use. By the time we’d pretty much cracked the OR mapping problem in Java, the world had moved on and it wasn’t only about relational any more. Two VMware based examples: we are looking at using Neo4J to underpin Hyperic; also looking for a non-relational solution for another data capture / gathering app where volume is overwhelming the relational approach. Rod encouraged the audience to check out the Spring Data project and some of the interesting work being done in Grails and GORM. We are trying to make it easier and more consistent for developers to work with non-relational stores from Spring / Java.

Most applications of real complexity in the next few years will use both relational and non-relational databases

Cloud is the answer?

Having more choices is exciting to developers, but a very expensive thing to administer. We are seeing system complexity increase. E.g. Load-balancer -> web server -> app server -> cache -> (database, messaging system, noSQL Store), with reporting and monitoring across the board.  We spend far too much on maintaining things operationally, let alone thinking about code maintenance costs. Complexity is inhibiting business agility and innovation. 

“Cloud is the answer” – a little over-optimistic perhaps, but, “Cloud may make this possible”.  Benefits we need to see from cloud computing include: more efficient use of resources (the end of massive over-provisioning); infrastructure (middleware) available as a service – cloud should offer e.g mongo service, redis service, … ; app servers should be just “part of the fabric” in the cloud – you don’t interact at that relatively low level. Other services would include things such as mobile gateways, SMS publishing, …

When Rod says “cloud” he is talking about both public and private cloud computing. Most people think about public cloud first. Public cloud very important but of the $ value of workloads migrating to cloud over the next few years, private cloud will account for the majority.

As a larger enterprise, it makes perfect sense to operate your own private cloud. If you are an SME, it’s insane to run your own data center – this end of the market will migrate to the public cloud.

There are three layers of the stack in cloud computing, and these evolved in a possibly surprising way. SaaS (apps as a service)-> PaaS (middleware as a service) -> IaaS (vms as a service). Most successful and best adopted layer is probably SaaS – salesforce, netsuite, etc.… very easy to get started here. In IaaS the strongest player is Amazon EC2. A great deal of emphasis has been placed on IaaS though it is still a relatively small market in the world of enterprise IT (< $1Bn, e.g. much smaller than Oracle’s database business). A lot of the wins occur in the PaaS layer, something that so far has been pretty much neglected. Still too much work to get an application deployed and running on EC2 (using IaaS directly). IaaS is a good model if you want to something “relatively weird” that doesn’t fit on traditional app servers etc. and you need to build from the OS up. But most of us develop to a “platform” – e.g. writing an application with Spring. We need the availability of certain HA facilities, data stores etc., but fundamentally we care about the programming model, not what it runs on.

The right level of abstraction for cloud computing for custom app development is PaaS. Will see PaaS grow to be a much much larger part of the pie than it is today.

There have been a number of recent moves in the PaaS space: Salesforce acquired Heroku (PaaS already big play in Ruby community); Google made improvements to GAE; Amazon released Elastic Bean Stalk; VMware released Cloud Foundry.

Rod then referenced the Cloud Developer’s Bill of Rights as some of the things we want in a PaaS solution. A lot of these rights relate to portability and echo traditional Spring values.

Cloud Foundry is a PaaS with some novel characteristics in the marketplace, well aligned with Spring values. Two things above all have been important to Spring – make things simpler (conceal complexity, make consistent); make things portable – wherever you want to run your app, the best way to make it portable is to build it with Spring. The values of simplicity and portability are more important than ever as we move towards cloud computing. PaaS is really about business agility. The benefits sought by customers concern simplicity of operations and of development. Portability becomes even more important. Today there is an even broader choice of runtimes to deploy to: what does the stack look like? How many servers etc.? These things are abstracted away from you and rightly so – therefore the programming model really needs to offer portability. Some things around PaaS that need to be addressed can’t be addressed by the programming model only though. For example, the ability to be able to run the PaaS environment itself wherever you need (in private data center, for local development, … etc.). A second benefit of Cloud Foundry is the services that are available. Consumption of middleware is increasingly as a service – administered centrally and shared across many applications. Cloud Foundry provides some out of the box services, and has extension points to add in others. Final key part of Cloud Foundry is the programming model – we are not enforcing a single programming model, you have a choice, and if you want you can integrate additional ones yourself. Other PaaS offerings tend to be closed source, single language. Cloud Foundry is a very big initiative for VMware – 2 years in development, very good integration with Spring as you would expect, integrated with our overall tool chain.

Developer tools and middleware need to be cloud ready. Developer productivity is more important than ever. One of our major focus areas is to enhance the way that you build apps and move them to production. Today most customers build apps using Spring or Grails, deploy onto vFabric Middleware products (e.g. tc server, GemFire, …).  Going forward there is a longer and more integrated tool chain to help maximize productivity.

The chain looks like this: Productivity tooling (use either Grails or Spring Roo)-> portable framework -> cloud based development (for e.g. Load, stress, penetration testing, CI, source control, issue tracking, …) -> vFabric middleware -> deployment destination (clouds).

“My opinion is that for most new projects you should be using either Grails or Spring Roo” for productivity. (In the context of developing Java apps for the enterprise).

We are building an experience that gives productivity and simplification across the lifecycle.

The next thing we will be releasing beyond Cloud Foundry is something you used to know as Code2Cloud, and is now likely to be called Cloud Foundry Code.

The experience: create a new project using STS, deploy to Cloud Foundry and have automatic source code hosting, issue tracking, CI set up for you. We will also enable vFabric middleware for consumption as a service both inside and outside the firewall.  Level of abstraction for app deployment is moving up from app server…

In Closing:

These are ambitious goals, and more innovation is still required. One of the key vehicles for innovation is open source. The majority of things talked about in this keynote are open source.  This year, the shift to the cloud *has* happened. What are you going to do to contribute and reap the benefits?

Cloud computing is one of the most important things for us as Java and Groovy developers to get to grips with. One of the key drivers is business agility and simplicity. Therefore everything we are doing is aiming to simplify the overall experience. PaaS will become more and more important.

Q: when will Code2Cloud (Cloud Foundry Code) be available? A: we expect to release by the end of June, in beta at the moment.

[Correction: Cloud Foundry Code is currently in private alpha, we expect to go into beta by the end of June]