RSS

Monthly Archives: October 2012

SDKs and an OpenStack Grizzly Summit Wrap Up

OpenStack Grizzly SummitThe OpenStack Grizzly Summit has wrapped up and it was another huge success. The new format of having the Summit and Conference at the same time worked well. Good venue, the wifi held up, the organizers accommodated the overflow people (i.e. 200-300 unregistered attendees) with amazing aplomb, there were always healthy food choices, and coffee/water was present at all times. All in all, great organization by the OpenStack Foundation that allowed everyone to stay focused on the task at hand, improving OpenStack in every way possible.

I wanted to highlight some of the progress made on the Software Development Kit (SDK) front during the Summit.

Monday

I led a session titled SDK Documentation Discussion (here are the raw etherpad notes). I’ve been participating in Summits since Bexar in San Antonio and, as far as I know, this is the first time SDKs have been discussed in any depth so it turned out to be more of a survey of existing SDKs and brainstorming session. We started with a definition of SDKs to make sure we were all on the same page. I asked what SDKs people were using and got a variety of answers. On the topic of what criteria to use to allow an SDK to claim OpenStack support we thought that one day the Foundation might have some sort of validation test suite to determine compatibility.

Finally getting around to the topic of documentation, we brainstormed some ideas for documenting method level APIs in an SDK native doc tool while still keeping that information consistent with existing OpenStack specs. We also talked about a number of ways to keep up to date with changes to the various APIs. I think I can summarize that as simply having to tune my receptors to all the different channels where API change notifications can be broadcast. When the question of what it means for an SDK to be considered feature complete came up, I was somewhat surprised to hear the opinion that documenting your feature support can be more important the “feature-completeness”. Upon a bit of reflection, this makes good pragmatic sense. As a developer many times all I want to know is if this tool is going to do what I want it to without having to get a developer environment setup and prototype something.

I’ve been purposefully a bit vague in my descriptions above because one of the action items to come out of the discussion was for me to start tracking this stuff on an OpenStack wiki page and I didn’t want to duplicate the information here. You can now find that page at wiki.openstack.org/SDKs.

Tuesday

On Tuesday I spoke to Simon Anderson, CEO of DreamHost, and explained how Rackspace was committing developers to support open source cross-cloud toolkits such as jclouds, libcloud, fog, and php-opencloud. I talked about how HP is also supporting Java jclouds and Dell is supporting Ruby fog. How if we all throw our weight behind select SDKs, the quality and support of those SDKs will be better and we can avoid fragmentation. Plus we’ll inherit all of the existing OpenStack support, documentation, sample code, and user bases in those projects. Simon perceived the benefits immediately and let me know his team at DreamHost would start looking at supporting those toolkits as well.

The DreamHost team are a great group of people and I’m really looking forward to any future collaborations.

Wednesday

Wednesday started out with the Rackspace media team releasing the video A Look at SDKs.

Troy Toman from Rackspace gave a morning keynote. Personally I found it quite inspiring. He described how Rackspace runs on OpenStack trunk minus 2 weeks. We’ve got Grizzly already! Troy showed how we can have a build of OpenStack ready to go in 47 minutes and prepared for production. He also talked about how Rackspace engineers will turn their attention back to the community now that we’ve got OpenStack in production. We’re committed to trusting the community. It was also very generous of Troy to highlight our SDKs and give the Developer Relations Group a shout out.

A design session titled Standardizing client & API capabilities (here are the raw etherpad notes) led by Gabriel Hurley I found to be very relevant to SDKs as well. He was proposing an effort to achieve consistency and common feature (e.g. pagination, filtering, bulk objects action, etc.) parity across the various APIs as well. In all of the subsequent API design sessions I attended I was sure to bring up Gabriel’s effort and in the hopes that all of the API design efforts underway would take his proposals into consideration sooner rather than later. If OpenStack can deliver a more consistent API experience, it will drastically simplify writing SDKs and provider a better overall developer experience.

Thursday

On Thursday I gave my Control the Clouds: Developer Experience with jclouds workshop. You can find the workshop itself at bit.ly/jclwork. Considering it was in the second last time slot session of the entire Summit, there was a relatively good turn out of 20-25 people. The workshop went off without any network problems (thanks Cisco) or Murphy’s Law style issues, which was a relief. I got some good questions at the end so I knew people were engaged.

By this point in the Summit the tech press were well aware of the new Rackspace open cloud SDKsthanks to the masterful efforts of Wayne Walls and Hart Hoover. They got the SDKs mentioned in the following articles.

Coda

A lot of hard work and preparation went into getting ready for this Summit. Everything went amazingly well and it exceeded my expectations in practically every way possible. Cheers to the OpenStack Foundation for putting on such a well organized event. No doubt the Grizzly release will live up to its namesake.

 
Leave a comment

Posted by on October 22, 2012 in jclouds, openstack, rackspace

 

Introducing the Rackspace Developer Relations Group and open cloud SDKs

ToolkitI’m proud to help announce and to be a part of the Rackspace Developer Relations Group as a Developer Advocate. The group has been around for a little while now and I only joined them a few of months ago but we’ve had our heads down building Software Development Kits (SDKs). Well now we’ve got a killer Java SDK and PHP SDK for the Rackspace open cloud ready to go and we’d like to let the world know. We’ve been introduced on the official Rackspace blog but I wanted to throw my 2 cents in here too.

What is an SDK?

The documentation for our SDKs begins with this Introduction, that page does a great job of defining Software Development Kits so I’m just going to highlight a numbers of things here. An SDK consists of the following components:

  • A set of language bindings that provide a language-level API for accessing cloud services (as opposed to forcing developers to use the REST/HTTP APIs directly) in a manner consistent with language standards
  • A Getting Started document that shows how to use the API to access the Rackspace public cloud
  • A detailed API reference document
  • Tested sample code that you can use as a “starter kit” for your own cloud applications

In effect we take a lot of the grunt work out of using the OpenStack and Rackspace APIs. Developers can get down to business in the language of their choice without having to worry about writing all of the “plumbing” code to access the REST/HTTP APIs directly. Our sample code and documentation accelerate their understanding of how to best utilize the SDKs so they can get up to speed quickly. The cloud is still relatively new and people need all the help they can get.

As you can probably tell from the other content on this blog, the Java SDK that I’ve been working on is powered by jclouds. Our PHP SDK is developed by Rackspace as an open source project.

For the official announcement checkout Rackspace Announces Open Cloud SDKs.

What is a Developer Advocate?

This is still a relatively new role in the software industry and people often ask me this question. My current answer is that I build and promote tools to make it easier for developers to create software on OpenStack and the Rackspace open cloud. I also ensure third-party developers have a Fanatical advocate within Rackspace.

The meaning of “Developer” is two-fold. First of all, I myself am a developer. Right now I’m building SDKs, writing sample code, and producing documentation that use the OpenStack API to communicate with the Rackspace open cloud. Being a developer and cutting code is essential to the role. The open source world is a meritocracy (rightfully so) and code and documentation speak louder than just flapping your gums. The second meaning refers to third-party developers creating software on OpenStack and the Rackspace cloud. And that brings us to…

The meaning of “Advocate”, which is also two-fold. Most importantly I’m an advocate within Rackspace for those third-party developers building solutions on OpenStack and the Rackspace cloud. This too is essential as those developers need a voice within Rackspace. Customers are at the heart of everything Rackspace does and this is just a natural extension of that. The second meaning refers to the fact that I also advocate for the tools I’ve built and OpenStack and the Rackspace open cloud too. Like any other developer, I want my software to be used and get feedback on how to improve it.

I love both the breadth and depth of this job. It covers a lot of ground. Here’s a sample of what we do:

  • design work
  • writing code
  • debugging HTTP layer issues
  • documenting
  • blogging
  • presenting at conferences
  • promoting our ecosystem
  • talking with developers
  • and the list goes on.

We also get the opportunity to dive deep into technical or customer issues. And sometimes it all happens on the same day.

What does the Developer Relations Group do?

The introduction on the Rackspace blog does a fine job of answering this so I’ll just highlight a few things here.

  • Lower developers’ barriers to entry
  • Help developers succeed
  • Increase developers’ ROI

We’ll achieve this through producing SDKs and Developer Advocacy. While doing so we need to be humble and helpful to the open source community at large. It’s not always easy balancing a businesses needs with those of an open source project but, if done in an open and transparent way, in the end it’s worthwhile. You’ll have a project/product that’s more featureful, has better adoption, and is more resilient to changes in the market.

Coda

Clearly I’m excited to be a part of this new group at Rackspace. There’s still a lot more work to do but I’m looking forward to it and that makes the mountain that much more fun to climb. Please give our SDKs a try and let us know what you think at sdk-support@rackspace.com.

 
Leave a comment

Posted by on October 15, 2012 in jclouds, openstack, rackspace, sdk

 

SDKs at the OpenStack Grizzly Summit

GrizzlySDKs are a vital resource for any ecosystem and SDKs for OpenStack are proliferating. The OpenStack Grizzly Summit has a few sessions dedicated to SDKs that I wanted to highlight.

  1. On Monday @ 11:00 am I’ll be leading the session SDK Documentation Discussion. Some of the topics/questions we’re going to address are:
    1. What SDKs are you using and why?
    2. How do we track SDKs that support OpenStack?
    3. Where do we track SDKs that support OpenStack?
    4. What criteria do we use to allow an SDK to claim OpenStack support?
    5. When documenting the SDK API at the function level, do you duplicate info from api.openstack.org or do you just link to it? Are there other options?
    6. What’s missing from the documentation of the SDKs you’re using?
  2. On Tuesday @ 11:50 am Rob Hirschfeld from Dell will be leading the session Quantum Fog! Networking for Programmatic Overlays
    1. Fog is the leading Ruby abstraction library for the OpenStack API and it’s embedded in several ecosystem products.  With the addition of Quantum, there is a need to extend Fog’s models to comprehend cloud networking.
  3. On Thursday @ 3:20 pm I’ll be leading the workshop Control the Clouds: Developer Experience with jclouds
    1. jclouds is the leading Java cross-cloud toolkit for the OpenStack API and already has a broad adoption base. In this workshop, you will learn how to write code that can control any cloud with jclouds.

Coda

Contributing to and supporting an open source SDK should not be taken lightly. The OpenStack community needs to consider throwing its weight behind a select number of SDKs to in order to provide the best developer experience, the highest possible quality/support, and to limit fragmentation. Please join us at the sessions above and let’s find out how the OpenStack community can achieve this.

 
Leave a comment

Posted by on October 10, 2012 in jclouds, openstack, sdk

 

Contributing OpenStack Support to jclouds

Guava

Update: Folsom instructions

jclouds is a popular cross-cloud toolkit that covers an impressive number of public and private cloud providers. OpenStack is among these providers and the OpenStack ecosystem benefits from good support in jclouds because it allows developers who are comfortable with Java or jclouds to easily adopt OpenStack and it enables hybrid cloud development. Likewise, jclouds benefits from good OpenStack support because OpenStack is a popular Infrastructure as a Service provider and already has deep roots in the open source community.

A number of times on the jclouds-dev mailing list and IRC I’ve seen developers who want to help contribute OpenStack support to jclouds but aren’t entirely sure about how to go about it. There are a number of things you need to get started.

  1. A jclouds development environment
  2. Some background on jclouds development
  3. An OpenStack test environment

jclouds dev env

To setup a jclouds development environment start at their Developer Resources page. From there the most relevant page is Contributing to the jclouds Code Base. If you’re an Eclipse user, you’ll want to check out Using Eclipse. Learning about Testing in jclouds is essential and the Additional References and Guides for Advanced Users also contains a wealth of design and implementation documentation. You might also want to browse through the jclouds-dev mailing list.

jclouds dev background

The foundation of jclouds is built upon Google’s general purpose library Guava and their lightweight dependency injection framework Guice. You don’t necessarily have to know these to start contributing but being aware of it and having the reference documentation on hand will be helpful. To talk to OpenStack’s RESTful web services jclouds uses the Java API for RESTful Web Services (JAX-RS).

OpenStack

For those unfamiliar with setting up an OpenStack dev/test env this can be a daunting task. There are a number of ways to do it and often it boils down to a developer’s preference on how they want to deploy it…so I’ll share my personal preference. :)

My OpenStack test env consists of VirtualBox running an Ubuntu 12.04 Server (Precise Pangolin) virtual machine with devstack. This guide assumes you’re comfortable installing Ubuntu on VirtualBox and know your way around the command line and git.

  1. Download and install VirtualBox
  2. The networking can be a bit tricky and I’ve already addressed this in the ServerFault question What is the correct network configuration for a devStack VM (virtualbox)?
  3. Download the 64-bit PC (AMD64) server install CD of Ubuntu 12.04 and create a VirtualBox VM with it
  4. Once it’s ready to go you should be able to login to it from your terminal
    1. ssh myusername@172.16.0.1
  5. Once logged in, take a break, and have a look at the instructions for devstack
  6. If you want a release branch of OpenStack (e.g. Essex, Folsom, etc.), you need to do things a little differently
    1. Essex:
    2. git clone https://github.com/openstack-dev/devstack.git -b stable/essex devstack/
    3. Folsom:
    4. git clone https://github.com/openstack-dev/devstack.git -b stable/folsom devstack/
    5. In the devstack/ directory create a localrc file that only uses release branches only
      1. Essex example
      2. Folsom example
  7. If you want the latest and greatest master branch of OpenStack, in the devstack/ directory, create a localrc file like this
  8. In both cases
    1. Change to your devstack directory
    2. Run stack.sh
    3. When it completes, configure Keystone to return the VM’s “public” IP
      1. Essex:
      2. sed -i "s#publicURL = http://10.0.2.15#publicURL = http://172.16.0.1#g" /etc/keystone/default_catalog.templates
      3. Folsom:
      4. mysql -uroot -pdevstack keystone
      5. update endpoint set extra = replace(extra, '"publicurl": "http://10.0.2.15', '"publicurl": "http://172.16.0.1') where instr(extra, '"publicurl": "http://10.0.2.15') > 0;
    4. Kill the existing screen session (that contains all of the OpenStack services)
      1. screen -X -S stack quit
    5. Restart screen (with all of the OpenStack services)
      1. screen -c stack-screenrc
  9. Outside of the VM, back on your local machine, you should now be able to follow the instructions in my post jclouds and OpenStack to verify that everything is working as expected
  10. Once you’ve verified it’s working, then port the jclouds and OpenStack example into your jclouds dev env and go from there

Coda

Clearly it’s non-trivial to get a full development/test environment like this up and running. However, the benefits are a healthier OpenStack ecosystem that’s inclusive of the many Java developers around the world.

This post was written on Mac OS X 10.7.4, Java 1.6.0, jclouds 1.5.1, and OpenStack Essex/master.

 
Leave a comment

Posted by on October 3, 2012 in devstack, java, jclouds, openstack

 
 
Follow

Get every new post delivered to your Inbox.