Does Infrastructure Matter to App Developers?

As Infrastructure becomes more commoditized and an extreme pricing war rages on at the Infrastructure as a Service (IaaS) layer, infrastructure vendors are moving up the stack towards offering more services that app developers can consume. Some of the emerging trends in modern application development makes infrastructure and application development go hand-in-hand. Especially with

  • Building applications for scale including Big Data
  • Agile processes and Devops including Platform as a Service
  • Cloud computing in general

Plus enterprises are still spending substantially for pure storage.

Having said all this I am moving on to EMC and help make the case that the company cares about application developers and to reduce the friction for developers to work with the EMC technologies and products. Although some of the core EMC products might seem far removed from the application developers, products from VMware, Pivotal and RSA are all extremely relevant to application developers.

Many of Spring commiters are from Pivotal. Spring, a developer favorite which also has significant adoption in the enterprise is the underpinnings for many of the Pivotal technologies and products.

EMC is also making a push with some anticipated announcements to the OpenStack community. Some of the core EMC products, like ViPR are making the move to be more open and developer friendly.

Hoping that our newly formed team @emccode can work with open source and developer communities, contribute to projects and solutions and show that EMC cares.

p.s. In late breaking news EMC buys Cloudscaling today and I had nothing to do with it :-)

Should App Developers care about OpenStack?

Will Havana break the Ice with Application Developers?

OpenStack Havana release is out. I realize that the term developer needs to be properly qualified. I am trying to draw the distinction from an OpenStack developer, which usually means a contributor to the OpenStack infrastructure in this context. An Application developer would leverage the OpenStack platform.  Whereas OpenStack is the life and blood of an OpenStack developer, does the Application developer even care about OpenStack?

The Application Developer has been at the center of the “App economy”, which was spurred by the increasing use of Apps on a smartphone and the “API economy”, which is being driven by the huge amounts of data being produced and consumed as web services. Both these economies have grown by leaps and bounds. Large enterprises have jumped on the bandwagon to be able to meet the demands of the consumers. With OpenStack the Application developer has remained in the periphery (so far).

There are definitely signs of OpenStack becoming more developer friendly with the Havana release. The Heat project, which includes AutoScaling, is aimed squarely at application development. Project Neutron provides many services, like Load Balancing, Firewalls and so on. Project Trove, which is a Database as a Service, Project Marconi, which is a Queuing service and Project Savanna, which is intended for Big Data processing are all incubated in the Icehouse release.

Does IaaS matter to Joe Developer?

Before my life as a Racker and OpenStacker, I was an application developer myself. I developed in Java/Java EE. I attended my first OpenStack summit at Portland and although I was very impressed by the phenomenal growth of the community and the participation of enterprise companies, the sessions I attended did not quite hit home the value proposition of OpenStack to an app developer. I needed something more than the argument that developers should care about IaaS since they don’t need to wait for days or weeks for IT to be able to standup their development, testing or production environments.

I decided to be proactive and rounded up some stalwarts from the OpenStack community who were also at similar crossroads and submitted a panel for the OpenStack summit at Hong Kong that got accepted. 

Let your voices be heard, ask the tough questions and act!

Should Developers care about OpenStack? Does the community need to grow to millions or is it fine where it is? Can it cross the chasm with respect to app developer adoption? What is the killer app. for OpenStack? Does OpenStack need an app. marketplace? What can the community do to entice more developers? Or should it?

These are some of the many questions I have heard frequently. I don’t believe that all of these questions will be answered in the panel, but hopefully it will spur some action from everyone in the community going forward. If you’re attending the summit, there will be ample opportunities to get some answers from the panelists. If you’re not at the summit and still care about OpenStack and the developer, tweet (@ragss) or email the question to me for possible use during the panel or in the future. Links to the slides are here.

Is NoSQL the next wave?

I attended a NoSQL talk last night at my local JUG and the interest level for NoSQL seems to be higher than ever before. Without getting into a religious discussion on why the name NoSQL and is it really anti SQL, modern applications like Facebook, NetFlix, Zynga and so on have different needs and use cases from traditional applications. It’s perhaps a cliche to say that scalability and performance is paramount. However, more importantly, the move towards NoSQL is also seen as a way to simplify the architecture whenever possible. It’s no panacea by any means either. Eric Brewer’s work in 1990s – CAP theorem talks about the tradeoffs that need to be made at a very high level. In addition, you have to deal with technologies that are nowhere near the level of maturity as SQL.

The discussions that ensued after the talk made it clear that the NoSQL movement has some really quick growing up to do to become an integral part of the enterprise. Data visualization tools, regulatory compliance requirements and even ETL tools for most of the vendors seem to be well short of expectations. From a developer viewpoint, there is very little in terms of standardization to ease the pain of migration. Language APIs even from the same vendor are not all equal. However, companies from tiny to huge are venturing seemingly undeterred into NoSQL for a multitude of reasons, not all of them technical.

How is it possible to answer the question — should I even consider a NoSQL system, if so, how do I go about evaluating which one, how do I get started and so on?

There are a number of webinars, conferences that you can attend which go through some of these war stories. Often times, it’s great to get the buzz from the peers. The QCon panel provides a good high level perspective.  There are a number of QCon conferences and NFJS conferences for a vendor neutral viewpoint. I personally am doing an O’Reilly webinar, speaking at the Cleveland JUG and at some of the Couch conferences, the next one happens to be in Portland, OR.

Hope to see you at some of these venues and talk about NoSQL, Java, Ruby or whatever development trends that you want to talk about.


The Java PaaS Rush – Crossing the Chasm?

As the National Football League (NFL) season opens up and teams are tweaking their rosters, I am pondering a different kind of PaaS rush.

The industry consensus is that the PaaS market in general is growing slowly and lagging the hype curve. However,  there is already a host of Paas vendors with some form of enterprise Java support,  just like in the hey days of the Java Application Servers. From Google with it’s App. Engine to even Microsoft making a play for Java developers on Microsoft Azure, there is a whole bunch of platforms including Amazon, CloudBees, Cloud Foundry, Cumulogic,, OpenShift and the list goes on.

I could make a detailed comparison of the different platforms (I’ll save that for JavaOne 2011). Instead, I am left marveling at how Java, or more accurately the Java  ecosystem, has been able to spur the recent growth of the PaaS market despite being considered old at a decade and half. No doubt that many of these PaaS platforms support other languages as well like Ruby, Python and so on, but it seems like the defacto choice for enterprise deployment is .jar, .war or .ear files, or some flavor of that. Consequently, PaaS vendors are opening up their platforms to Java developers including supporting the APIs that Java developers are familiar with and hope that they come in droves when they make a move to the cloud.

OScon Java had it’s share of talks on Java. The “Good, Bad, and Ugly of Java” talk went into how the designers of Java mostly made good choices in the initial design of the language that has withstood the test of time and made huge inroads into the enterprise. The talk by Twitter about using the JVM for better performance and to tap into a large developer population while keeping the operations folks happy at the same time since they understood JAR files, how to interpret GC logs, etc. goes to say how important are the aspects of familiarity and not introducing something fundamentally new within the enterprise.

Given a choice between agility and familiarity enterprise developers pick familiarity for a variety of reasons. Of course if you can add in agility as well that the cloud in general and PaaS  in particular provides to an existing environment, it becomes a very compelling value proposition to enterprises.

It would be disingenuous to claim that Java is the only agent that might facilitate the crossing of the PaaS chasm. The gradual embrace of Open Source in the enterprise, the variety of technological improvements and a motivation to control costs will help in the growth of PaaS. Enterprises realize that getting to PaaS i.e. migrating apps. to PaaS is expensive and time consuming and are holding off on migrating to PaaS en mass. This could be substantially eased by significantly reducing the learning curve and not having to introduce entirely new design paradigms.

Many of the PaaS vendors are taking the familiar application development lifecycle and products (like the Tomcat server) that Java developers know and love and adapting it to the cloud. From providing a version control system (like git or svn) to being able to analyze logs, deploy newer versions and even rollback to earlier versions all with a few clicks, the goal seems to become a part of the PaaS rush that Java and other developers are poised to make.

Is Cloud Computing on the developer RADAR? Really?

As kids head back to school, I went back to school as well. I was extended an invitation to attend the No Fluff Just Stuff conference which is a small and compact conference where everyone seems to know everyone else. The attendees all seemed to be hard core developers or architects who seemed to be in tune with the future but mostly focussed on the present.

The sessions I attended were varying in technical depth, but, mostly had stuff and minimal fluff. I did manage to pick up some great nuggets. It was definitely worth my time. I was at a 60-minute expert panel where questions ranged from the future of Java after Sun is consumed by Oracle, future of Java 7, scripting languages, deployment, EJBs, Rich Internet Applications including Flex and JavaFX and the gamut.

What surprised me most was that there was not a single question on the cloud and the word was barely even mentioned. It did come up once as a substitute term for the internet.

While Amazon and Infrastructure as a Service has made a huge dent on startups and small companies and IT organizations for the ability to outsource the data center, I think the cloud has made a very minimal impact on the developer mindshare — for now anyway.

The reasons for this are certainly varied. Developers are probably waiting for the hype to die down to make an assesment of the reality. Unlike evolving distributed system models, from socket programming, to RMI, to web services and so on, the cloud does not seem to be introducing a new programming paradigm. The developer tools are still evolving and there does not seem to be a set of APIs for the cloud other than the Platform as a Service which are relevant to that particular platform alone.

In my informal conversations with the attendees, there were lot of other things besides the cloud that occupied their minds. So, the question that was foremost in my mind after attending the conference was “Is Cloud Computing on the developer RADAR? Really?” Or was it too small a sample size to be representative?

What about Data Latency and cloud computing?

Over a decade back when we were still in the CORBA days, Peter Deutsch came up with the fallacies of distributed computing. Many distributed systems have been flawed due to willfully ignoring some of the fallacies and expecting technologies to obviate some of the inherent limitations in distributed computing. The premise was that those developers and designers who ignored these fallacies would in effect end up with a distributed system that would have some serious limitations in its functionality sooner or later. It was best to plan for some of these problems which was (and still is) inherent in distributed computing.

One of these fallacies is that “Latency is zero“. In traditional computing, the compute and data was typically hosted on the same system and the data latency was determined by the storage disks and the data bus speeds.  It was a simple matter of buying better hardware to overcome data latency if it was ever an issue. In cloud computing and especially when we get to network of clouds with data expected to flow around different clouds, latency (however minimal it is) could be an  issue depending on the data being manipulated, the network speeds and so on. Add to this the fact that the entire data or part of the data should be encrypted and decrypted when it moves around unreliable and public networks, and the fact that data needs to be streamed, latency will soon add up and could become a serious issue.

In the web era, many companies like Akamai have specialized in making data available closer to use and minimizing network latency. Some of these and other companies are already looking into this issue vis-a-vis the cloud, but, just the other day when I was twiddling my thumbs waiting for a not-so-big file to upload from my desktop to my EC2 machine instance, I was wondering how many of those dabbling with cloud computing would eventually be faced with the same question that I did, “How does Data Latency affect my cloud solution or design?”.

What is the most important criterion in picking your cloud vendor?

With the recent launch of the Blackberry App World program in line with Apple’s App Store and Microsoft expected to launch it’s own app store in the not too distant future companies are expecting developers to have their own skin in the game. The Blackberry App World program requires a registration fee of 200.00 USD which is no chump change. In return, freelance developers are able to market their applications to the global customer base that they typically would have difficulties reaching without a huge marketing budget. In the ideal scenario, as more applications become available and more users start using it, more developers start participating, creating what I heard as the “flywheel effect” from Wener Vogels, CTO at The faster this flywheel turns, the sooner it will result in a significant drop of prices for everyone involved.

In my previous blog I alluded to the different taxonomies of cloud computing and how as a developer Platform as a Service might be more interesting. There are more 1-off applications in an Infrastructure as a Service (IaaS) cloud rather than a Platform as a Service (PaaS) cloud. However, applications deployed in a PaaS cloud are more focussed on solving a (or a set) of business problem(s) for a typically large community of users. Drawing a comparison to the  app stores,  the goal of Platform as a Service is to offer a superior platform (technology matters, but, is not the main focus)  that helps leverage the ecosystem and the community.

How important is it for you application to leverage an existing platform and an existing user base? Is it important enough that it might become the sole criterion to choosing a particular cloud vendor?


Get every new post delivered to your Inbox.