RSS

Category Archives: openstack

jclouds Manual Authentication with OpenStack

LockI got a comment on my jclouds and OpenStack post about how to get the Tenant ID via jclouds. The first solution that popped to my mind was to get the Tenant ID out of the access data structure that gets returned upon authentication. This turned out to be a bit trickier than expected as jclouds automatically authenticates on your first actual interaction with a cloud (i.e. the very first time you call a method that needs to talk to the cloud). To get the access data structure I needed to manually authenticate. Here’s how to do it.

Get jclouds

First off, get the jclouds JAR files.

  1. Ensure you are using the Java Development Kit (JDK) version 6 or later.
    • javac -version
  2. Ensure you are using Maven version 3 or later.
    • mvn -version
  3. Create a directory to try out jclouds.
    • mkdir jclouds
    • cd jclouds
  4. Make a local copy of this pom.xml file in the jclouds directory.
    • mvn dependency:copy-dependencies "-DoutputDirectory=./lib"
  5. You should now have a directory with the following structure:
    • jclouds/
      • pom.xml
      • lib/
        • *.jar

Manual Authentication

Put this code in your jclouds directory to do manual authentication. An explanation follows.

  1. Compile.
    1. javac -cp ".:lib/*" JCloudsManualAuthentication.java
  2. Run on OpenStack.
    1. java -cp ".:lib/*" JCloudsManualAuthentication openstack-nova demo:demo devstack http://123.123.123.123:5000/v2.0/
  3. Run on Rackspace as this code works equally well with any OpenStack powered provider.
    1. java -cp ".:lib/*" JCloudsManualAuthentication rackspace-cloudservers-us myRackspaceUsername myRackspaceAPIKey

The key to the example is lines 93 & 94. Because jclouds normally handles the authentication manually we have to use a bit of Guice-fu on line 93 to get a hold of the Function that can manually authenticate by taking our Credentials and returning the Access data structure. On line 94 we execute that Function and the authentication happens. After that we can run through the Access object and get the information we need.

Coda

It’s rare that you would need to authenticate manually in jclouds. Most of the time letting jclouds automatically handle the auth for you suffices. However, if you do need to authenticate manually, this method will give you a lot of information about your OpenStack environment.

 
Leave a comment

Posted by on June 17, 2013 in jclouds, openstack, rackspace

 

The OpenStack Community Welcomes Developers in All Programming Languages

The OpenStack community has been and needs to continue to be a welcoming community for developers in all programming languages. Naturally I’m referring to developers who are building systems on top of OpenStack and not the developers of OpenStack itself.

OpenStack Nation

This post is prompted by a minor incident in the #openstack IRC channel. I’m not looking to single people out so I’ll use a pretty generic description so it can’t so easily be found in the IRC logs.

A developer came to #openstack to ask a question about a software development kit (SDK) in another programming language. Within 1 minute he got a reply that can only be described as snarky. Undeterred, he went ahead and asked his questions. 20 minutes later a couple of more snarky responses were added to it. No real help at all. It’s not the lack of help that’s at issue though. It’s the unwelcoming attitude.

I have not seen that developer in the channel since then.

Like I said, a minor incident. I don’t want to blow this out of proportion but it does need to be addressed. It’s one of those cases where you see the cracks start to appear, it’s best to fix them right away before they become real problems.

I’m sure we’ve all been part of such chats about languages. When you’re face-to-face or online but know the people personally, it usually goes without saying that it’s good natured. However, when you’re new to a community, it’s not so clear. Of course the OpenStack community is Python-centric but the OpenStack API is not. We need developers from all of the other languages building on top of OpenStack in whatever language they need to work with. Remember, it might not even be their choice!

Coda

Let’s continue to be good stewards of the OpenStack API and encourage it’s use by all programming languages by being an inclusive and welcoming community.

If you ever encounter someone looking for help with another language, you can always point them to the SDKs wiki page. They should be able to find their way from there.

 
3 Comments

Posted by on June 12, 2013 in openstack

 

Swift/Cloud Files Cross Origin Resource Sharing Container with jclouds

Cross-origin resource sharing (CORS) is a mechanism that allows Javascript on a web page to make XMLHttpRequests to another domain, not the domain the Javascript originated from. Such “cross-domain” requests would otherwise be forbidden by web browsers, per the same origin security policy. CORS defines a way in which the browser and the server can interact to determine whether or not to allow the cross-origin request. It is more powerful than only allowing same-origin requests, but it is more secure than simply allowing all such cross-origin requests. [1]

In the context of OpenStack Swift and Rackspace Cloud Files a CORS enabled container allow users to upload files from one website–or origin–to your Swift/Cloud Files container. When you set the CORS headers on your container, you tell it which sites may post to your account, how often your container checks its allowed sites list, and whether or not metadata headers can be passed with the objects.

You can find example code to enable and configure CORS at CrossOriginResourceSharingContainer.java. To allow users temporary access to a such a container you would use example code like GenerateTempURL.java.

[1]
http://en.wikipedia.org/wiki/Cross-origin_resource_sharing

 
Leave a comment

Posted by on June 4, 2013 in jclouds, openstack, rackspace

 

We Wrote an OpenStack Operations Guide in 5 Days!

OpenStack Operations GuideAnd here it is…the OpenStack Operations Guide. You can read it in EPUB, MOBI, PDF, or print. The electronic formats are free to download so click away. The print version costs $29.90 and all proceeds go to the OpenStack Foundation to support more book sprints like the one that produced this book.

To be honest it was an exhausting and sometimes stressful week. The stress was purely a product of trying to produce a book in 5 days. The co-authors of the book were professional, knowledgable, and just plain fun to work with at all times. There was lots of discussion and the occasional disagreement but nothing that we couldn’t resolve amongst ourselves amicably. Probably the biggest disagreements were over hyphenation and em dashes.

Thanks!

It was a genuine privilege to work with these people.

  • Diane Fleming: OpenStack Docs Writer at Rackspace
  • Tom Fifield: Cloud Architect and Operator at NeCTAR
  • Anne Gentle: OpenStack Doc Lead at Rackspace
  • Lorin Hochstein: Cloud Architect and Operator at Nimbis Services
  • Adam Hyde: Book Sprint Facilitator at FLOSS Manuals (founder)
  • Jonathan Proulx: Cloud Architect and Operator at MIT
  • Joe Topjian: Cloud Architect and Operator at Cybera

We also had a number of great support staff from the Austin Texas Rackspace office who are thanked in the first chapter of the book.

What Next?

Now that we’ve written book it doesn’t stop there. As everyone knows, as soon as technical books are published, they’re out of date. With OpenStack Grizzly right around the corner and the amazing pace of development on OpenStack, that certainly applies to this book. To combat the bit rot we’re working out the detail to allow anyone to contribute to the book. We also plan on getting back together at the design summits and updating the book to the most recent stable release of OpenStack.

Coda

Would I do it again?

Yep.

 
Leave a comment

Posted by on March 7, 2013 in openstack, operations

 

We’re Writing an OpenStack Operations Manual in 5 Days!

OpenStackAnne Gentle, the OpenStack Doc Lead and Technical Committee member, must be crazy. She’s proposed that a group of us write an OpenStack Operations Manual in 5 days. But there’s a method to her madness. She’s assembled a group of long-time OpenStack operators and people known for their documentation skill. She’s handled the logistics of getting all of these people from across the world together in the same room for 5 days. She’s recruited an experienced mediator and open source manual maker to assist us during the book sprint. And it all happens next week. She may be crazy but we believe in her and we can make this happen.

At the end of the 5 days we aim to have an OpenStack Operations Manual ready for download and print. OpenStack development moves very quickly but operators still need a jumping on point. A foundation of knowledge that they can use to build upon as OpenStack evolves. This manual aims to provide that sort foundation. It’s going to be an intense and probably stressful experience writing a book in such a short period of time but I know it’s worthwhile. Please take 5 seconds and vote for our panel On Writing the OpenStack Operations Manual in 5 Days at the upcoming OpenStack Summit to let us tell you how we did.

 
Leave a comment

Posted by on February 21, 2013 in openstack

 

Swift Only with OpenStack DevStack on the Rackspace Cloud

devstackSwift is not installed by default when using OpenStack DevStack. Sometimes you want to install Swift only for some testing without all of the other services getting in the way. Here’s how to setup Swift (and Keystone for authentication) with DevStack on the Rackspace Cloud.

Instructions

  1. If you don’t have one already, Sign Up for an open cloud account
  2. Go to the Cloud Control Panel
  3. Click Create Server
    1. Name: devstack-swift
    2. Image: Ubuntu 12.04 LTS
    3. Size: 4GB RAM (you could probably get away with 2 GB)
  4. Click Create Server and note the password and IPv4 address (when it appears)
  5. When your server is Active, switch to a Terminal and run the following commands to create the stack user.
    1. ssh root@<IPv4 Address>
    2. adduser --gecos "" stack
    3. adduser stack sudo
    4. grep -q "^#includedir.*/etc/sudoers.d" /etc/sudoers || echo "#includedir /etc/sudoers.d" >> /etc/sudoers
    5. ( umask 226 && echo "stack ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/50_stack_sh )
    6. exit
  6. I create the stack user like this because I’ve found that there are permission issues with screen if you create a “dummy” stack user and just switch to the stack user from the root user.
  7. Now login as the stack user and setup Swift:
    1. ssh stack@<IPv4 Address>
    2. sudo apt-get -y update
    3. sudo apt-get -y install git
    4. git clone https://github.com/openstack-dev/devstack.git
    5. cd devstack
    6. vim localrc # copy in the contents of this one
    7. ./stack.sh
    8. screen -r stack
  8. When running stack.sh you might see an error message that reads “ERROR: at least one rpc backend must be enabled”. Don’t worry about it, Swift/Keystone doesn’t need an rpc (AMQP) backend. You can also ignore any ImportErrors.
  9. When DevStack is done you can point your OpenStack clients and jclouds dev env all at <IPv4 Address>
  10. When you’re done with your development/testing you can delete the server to save money and just start fresh next time

Coda

Just a quick post to get Swift only up and running quickly!

 
2 Comments

Posted by on February 18, 2013 in devstack, openstack, rackspace

 

My (Very) Minor Contribution to OpenStack Bug Squashing Day

Ambush BugI wanted to somehow contribute to the OpenStack Bug Squashing today but I wasn’t sure what bug to squash. I didn’t have much time to participate so it had to be something I could complete within a couple of hours from scratch. I personally know several members of the OpenStack doc team so I checked out the OpenStack Manuals bug list to see where I could help. Anne helpfully suggested taking a look at this bug.

Documentation is an important (and often neglected) part of any technology. Many are quick to append “…but the docs suck.” to any sentence about a technology, especially open source. So let’s try to do something about it. Raise a bug, fix a bug, review a bug…everything helps.

Plus I think that Developer Advocates should be knowledgable about documentation technologies. Not necessarily experts but they should at least know how to “make it go.” Being able to build the docs and knowing how to make a quality contribution can go a long way here.

Coda

Okay. So I really only fixed one little bug. But the important part to me is that I learned a bit about the doc tool set, it forced me to build the docs, change a doc, verify the change, and submit it for review. Stuff I had never done for the OpenStack Manuals before. Now I’ve paved the way for contributions in the future!

 
Leave a comment

Posted by on December 13, 2012 in openstack

 

jclouds 1.5.3 Released With OpenStack Cinder and Rackspace Cloud Block Storage Support

jclouds 1.5.3 was officially released over the weekend. Another solid release from the jclouds community. Among a number of new features was the support I added for OpenStack Cinder and Rackspace Cloud Block Storage.

OpenStack Cinder

Cinder BlockCinder is the OpenStack block storage service. On physical machines, examples of block storage devices are hard drives and USB sticks. Cinder allows you to create persistent virtual block storage devices as volumes that can be attached to servers and mounted with filesystems. This means you can store any data on the volumes and that data will persist even if your server is terminated.

From an OpenStack API perspective there’s some historical context that will help you understand Cinder. Cinder was born out of the need to create an independent block storage service. However, originally, there was a block storage service in OpenStack called nova-volume that had its own API but was officially part of the Nova API. A new service for Cinder in OpenStack Folsom means a new Cinder API but this API was an exact copy of the nova-volume API in OpenStack Essex. This was done very deliberately to maintain backwards compatibility and ease the transition from nova-volume to Cinder. Both APIs now exist in Folsom but the Cinder API is the path forward where all new efforts will be focused.

Rackspace Cloud Block Storage

Cloud Block StorageCloud Block Storage (CBS) is Rackspace’s deployment of Cinder. This means it’s the Rackspace service that allows you to create volumes via the Cloud Control Panel, the CBS (Cinder) API, or jclouds! You can create standard volumes that work well for your everyday file system needs, and SSD volumes that deliver even higher performance for databases and other I/O-intensive applications. It also allows you to easily create a snapshot of your existing volumes that effectively act as a point-in-time backup.

To get started with jclouds and Cloud Block Storage check out the Getting Started: The Rackspace open cloud documentation, specifically the Working with Cloud Block Storage section. There are plenty of examples in the cloudblockstorage package of the Rackspace examples on how to use this essential new service. For instance, the CreateVolumeAndAttach.java is an end to end example of creating a volume, attaching it to a server, putting a filesystem on it, and mounting it for use to store persistent data.

Coda

Block storage is one of the fundamental building blocks (no pun intended) of computing. This has been an often requested feature for Rackspace and it’s great to help other developers use it by providing support for Cloud Block Storage and Cinder in jclouds. I encourage you to upgrade to jclouds 1.5.3 and checkout these new features.

 
Leave a comment

Posted by on November 20, 2012 in openstack, rackspace

 

OpenStack devstack on the Rackspace open cloud

devstackI’ve written about deploying devstack before in Contributing OpenStack Support to jclouds. That post covered running devstack locally in VirtualBox. I’ve been using that method for a while now but with a devstack env in VirtualBox, a jclouds dev env, browser, chat client, mail client, etc. etc. etc. all running on the same laptop it gets bogged down. I’m pretty sure I heard gears grinding and smoke was pouring out of the laptop at one point. I just don’t have the cores, RAM, and RPMs to run everything smoothly and you need smoothly or development starts to suck.

The biggest draw on resources is VirtualBox so it’s long past time to move that devstack VM to the cloud! Naturally I’m going to deploy on the Rackspace open cloud. It turns out this is significantly simpler/easier to do than deploying on VirtualBox.

Instructions

  1. If you don’t have one already, Sign Up for an open cloud account
  2. Go to the Cloud Control Panel
  3. Click Create Server
    1. Name: devstack-folsom
    2. Image: Ubuntu 12.04 LTS
    3. Size: 4GB RAM (you could probably get away with 2 GB)
  4. Click Create Server and note the password and IPv4 address (when it appears)
  5. When your server is Active, switch to a Terminal and run the following commands:
    1. ssh root@<IPv4 Address>
    2. apt-get update; apt-get install -y git
    3. git clone https://github.com/openstack-dev/devstack.git -b stable/folsom devstack/
    4. cd devstack/
    5. vim localrc # copy in the contents of this one
    6. ./stack.sh
  6. When devstack is done you can point your browser, OpenStack clients, and jclouds dev env all at <IPv4 Address>
  7. When you’re done with your development/testing you can delete the server to save money and just start fresh next time

Next

The next obvious thing to do is to automate the whole process above. I haven’t decided whether to do my own custom script or dive into something like CloudEnvy. Either way it needs to be automated but it’s always good to have the manual process documented to fall back on and understand everything end to end.

Coda

As you can see there are far fewer steps to creating a devstack env like this than on VirtualBox. There’s much less fiddling around with networking too. The big benefit for me though was the immediate increase in speed for the rest of my development tools on laptop.

 
6 Comments

Posted by on November 8, 2012 in devstack, jclouds, openstack, rackspace

 

SDKs and an OpenStack Grizzly Summit Wrap Up

OpenStack Grizzly SummitThe OpenStack Grizzly Summit has wrapped up and it was another huge success. The new format of having the Summit and Conference at the same time worked well. Good venue, the wifi held up, the organizers accommodated the overflow people (i.e. 200-300 unregistered attendees) with amazing aplomb, there were always healthy food choices, and coffee/water was present at all times. All in all, great organization by the OpenStack Foundation that allowed everyone to stay focused on the task at hand, improving OpenStack in every way possible.

I wanted to highlight some of the progress made on the Software Development Kit (SDK) front during the Summit.

Monday

I led a session titled SDK Documentation Discussion (here are the raw etherpad notes). I’ve been participating in Summits since Bexar in San Antonio and, as far as I know, this is the first time SDKs have been discussed in any depth so it turned out to be more of a survey of existing SDKs and brainstorming session. We started with a definition of SDKs to make sure we were all on the same page. I asked what SDKs people were using and got a variety of answers. On the topic of what criteria to use to allow an SDK to claim OpenStack support we thought that one day the Foundation might have some sort of validation test suite to determine compatibility.

Finally getting around to the topic of documentation, we brainstormed some ideas for documenting method level APIs in an SDK native doc tool while still keeping that information consistent with existing OpenStack specs. We also talked about a number of ways to keep up to date with changes to the various APIs. I think I can summarize that as simply having to tune my receptors to all the different channels where API change notifications can be broadcast. When the question of what it means for an SDK to be considered feature complete came up, I was somewhat surprised to hear the opinion that documenting your feature support can be more important the “feature-completeness”. Upon a bit of reflection, this makes good pragmatic sense. As a developer many times all I want to know is if this tool is going to do what I want it to without having to get a developer environment setup and prototype something.

I’ve been purposefully a bit vague in my descriptions above because one of the action items to come out of the discussion was for me to start tracking this stuff on an OpenStack wiki page and I didn’t want to duplicate the information here. You can now find that page at wiki.openstack.org/SDKs.

Tuesday

On Tuesday I spoke to Simon Anderson, CEO of DreamHost, and explained how Rackspace was committing developers to support open source cross-cloud toolkits such as jclouds, libcloud, fog, and php-opencloud. I talked about how HP is also supporting Java jclouds and Dell is supporting Ruby fog. How if we all throw our weight behind select SDKs, the quality and support of those SDKs will be better and we can avoid fragmentation. Plus we’ll inherit all of the existing OpenStack support, documentation, sample code, and user bases in those projects. Simon perceived the benefits immediately and let me know his team at DreamHost would start looking at supporting those toolkits as well.

The DreamHost team are a great group of people and I’m really looking forward to any future collaborations.

Wednesday

Wednesday started out with the Rackspace media team releasing the video A Look at SDKs.

Troy Toman from Rackspace gave a morning keynote. Personally I found it quite inspiring. He described how Rackspace runs on OpenStack trunk minus 2 weeks. We’ve got Grizzly already! Troy showed how we can have a build of OpenStack ready to go in 47 minutes and prepared for production. He also talked about how Rackspace engineers will turn their attention back to the community now that we’ve got OpenStack in production. We’re committed to trusting the community. It was also very generous of Troy to highlight our SDKs and give the Developer Relations Group a shout out.

A design session titled Standardizing client & API capabilities (here are the raw etherpad notes) led by Gabriel Hurley I found to be very relevant to SDKs as well. He was proposing an effort to achieve consistency and common feature (e.g. pagination, filtering, bulk objects action, etc.) parity across the various APIs as well. In all of the subsequent API design sessions I attended I was sure to bring up Gabriel’s effort and in the hopes that all of the API design efforts underway would take his proposals into consideration sooner rather than later. If OpenStack can deliver a more consistent API experience, it will drastically simplify writing SDKs and provider a better overall developer experience.

Thursday

On Thursday I gave my Control the Clouds: Developer Experience with jclouds workshop. You can find the workshop itself at bit.ly/jclwork. Considering it was in the second last time slot session of the entire Summit, there was a relatively good turn out of 20-25 people. The workshop went off without any network problems (thanks Cisco) or Murphy’s Law style issues, which was a relief. I got some good questions at the end so I knew people were engaged.

By this point in the Summit the tech press were well aware of the new Rackspace open cloud SDKsthanks to the masterful efforts of Wayne Walls and Hart Hoover. They got the SDKs mentioned in the following articles.

Coda

A lot of hard work and preparation went into getting ready for this Summit. Everything went amazingly well and it exceeded my expectations in practically every way possible. Cheers to the OpenStack Foundation for putting on such a well organized event. No doubt the Grizzly release will live up to its namesake.

 
Leave a comment

Posted by on October 22, 2012 in jclouds, openstack, rackspace

 
 
Follow

Get every new post delivered to your Inbox.