RSS

We Wrote an OpenStack Operations Guide in 5 Days!

07 Mar

OpenStack Operations GuideAnd here it is…the OpenStack Operations Guide. You can read it in EPUB, MOBI, PDF, or print. The electronic formats are free to download so click away. The print version costs $29.90 and all proceeds go to the OpenStack Foundation to support more book sprints like the one that produced this book.

To be honest it was an exhausting and sometimes stressful week. The stress was purely a product of trying to produce a book in 5 days. The co-authors of the book were professional, knowledgable, and just plain fun to work with at all times. There was lots of discussion and the occasional disagreement but nothing that we couldn’t resolve amongst ourselves amicably. Probably the biggest disagreements were over hyphenation and em dashes.

Thanks!

It was a genuine privilege to work with these people.

  • Diane Fleming: OpenStack Docs Writer at Rackspace
  • Tom Fifield: Cloud Architect and Operator at NeCTAR
  • Anne Gentle: OpenStack Doc Lead at Rackspace
  • Lorin Hochstein: Cloud Architect and Operator at Nimbis Services
  • Adam Hyde: Book Sprint Facilitator at FLOSS Manuals (founder)
  • Jonathan Proulx: Cloud Architect and Operator at MIT
  • Joe Topjian: Cloud Architect and Operator at Cybera

We also had a number of great support staff from the Austin Texas Rackspace office who are thanked in the first chapter of the book.

What Next?

Now that we’ve written book it doesn’t stop there. As everyone knows, as soon as technical books are published, they’re out of date. With OpenStack Grizzly right around the corner and the amazing pace of development on OpenStack, that certainly applies to this book. To combat the bit rot we’re working out the detail to allow anyone to contribute to the book. We also plan on getting back together at the design summits and updating the book to the most recent stable release of OpenStack.

Coda

Would I do it again?

Yep.

 
Leave a comment

Posted by on March 7, 2013 in openstack, operations

 

We’re Writing an OpenStack Operations Manual in 5 Days!

21 Feb

OpenStackAnne Gentle, the OpenStack Doc Lead and Technical Committee member, must be crazy. She’s proposed that a group of us write an OpenStack Operations Manual in 5 days. But there’s a method to her madness. She’s assembled a group of long-time OpenStack operators and people known for their documentation skill. She’s handled the logistics of getting all of these people from across the world together in the same room for 5 days. She’s recruited an experienced mediator and open source manual maker to assist us during the book sprint. And it all happens next week. She may be crazy but we believe in her and we can make this happen.

At the end of the 5 days we aim to have an OpenStack Operations Manual ready for download and print. OpenStack development moves very quickly but operators still need a jumping on point. A foundation of knowledge that they can use to build upon as OpenStack evolves. This manual aims to provide that sort foundation. It’s going to be an intense and probably stressful experience writing a book in such a short period of time but I know it’s worthwhile. Please take 5 seconds and vote for our panel On Writing the OpenStack Operations Manual in 5 Days at the upcoming OpenStack Summit to let us tell you how we did.

 
Leave a comment

Posted by on February 21, 2013 in openstack

 

Swift Only with OpenStack DevStack on the Rackspace Cloud

18 Feb

devstackSwift is not installed by default when using OpenStack DevStack. Sometimes you want to install Swift only for some testing without all of the other services getting in the way. Here’s how to setup Swift (and Keystone for authentication) with DevStack on the Rackspace Cloud.

Instructions

  1. If you don’t have one already, Sign Up for an open cloud account
  2. Go to the Cloud Control Panel
  3. Click Create Server
    1. Name: devstack-swift
    2. Image: Ubuntu 12.04 LTS
    3. Size: 4GB RAM (you could probably get away with 2 GB)
  4. Click Create Server and note the password and IPv4 address (when it appears)
  5. When your server is Active, switch to a Terminal and run the following commands to create the stack user.
    1. ssh root@<IPv4 Address>
    2. adduser --gecos "" stack
    3. adduser stack sudo
    4. grep -q "^#includedir.*/etc/sudoers.d" /etc/sudoers || echo "#includedir /etc/sudoers.d" >> /etc/sudoers
    5. ( umask 226 && echo "stack ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/50_stack_sh )
    6. exit
  6. I create the stack user like this because I’ve found that there are permission issues with screen if you create a “dummy” stack user and just switch to the stack user from the root user.
  7. Now login as the stack user and setup Swift:
    1. ssh stack@<IPv4 Address>
    2. sudo apt-get -y update
    3. sudo apt-get -y install git
    4. git clone https://github.com/openstack-dev/devstack.git
    5. cd devstack
    6. vim localrc # copy in the contents of this one
    7. ./stack.sh
    8. screen -r stack
  8. When running stack.sh you might see an error message that reads “ERROR: at least one rpc backend must be enabled”. Don’t worry about it, Swift/Keystone doesn’t need an rpc (AMQP) backend. You can also ignore any ImportErrors.
  9. When DevStack is done you can point your OpenStack clients and jclouds dev env all at <IPv4 Address>
  10. When you’re done with your development/testing you can delete the server to save money and just start fresh next time

Coda

Just a quick post to get Swift only up and running quickly!

 
2 Comments

Posted by on February 18, 2013 in devstack, openstack, rackspace

 

Netflix Aims to be Scalable, Functional, and Portable (or How to Run a Meetup)

13 Feb

NetflixOSSNetflix has been open sourcing the software that makes up their platform at a torrential pace. On February 6th, 2013 I attended their first ever open house where they gave developers a bit more insight into their motivation for going open source. The whole event was very professionally run and could be used as a template for how to run a meetup with that much content.

The Meetup

Upon entering the Netflix offices in Los Gatos, CA it was immediately obviously where to go to register. Two people were taking care of the registration and handing out name tags with names already printed on them. After registering, two people were passing out Netflix t-shirts and another two were directing traffic. Attendees were moved through the whole process smoothly and quickly. I helped myself to a drink and some popcorn and took my seat in the auditorium.

The meetup itself was broken down into 3 sections.

  1. Overview (30 min.)
  2. Lightning talks (30 min.)
  3. Project walkthroughs/demos (30 min. + however long you wanted to stay)

The overview was conducted by Adrian Cockroft, I’ll discuss that more below. Following the overview was a series of lightning talks about a select number of the open source projects which were presented by the developers themselves. As it should be, no lightning talk was longer than 5 minutes. Unsurprisingly the quality of the lightning talks varied from developer to developer. It was apparent that each developer had created their own slides. That’s okay but I think this aspect of the meetup would have benefitted from someone giving the developers a few tips on avoiding some common presentation anti-patterns (small fonts, too much text, etc.).

After the lightning talks were done we had a decision to make. Stay in the auditorium and get an introduction to Asgard or walk over to another area where the developer leads would be giving demos and walkthroughs of the various projects introduced during the lightning talks. I opted to check out the demos and walkthroughs and get a bite to eat.

The Overview

Adrian Cockroft first walked everyone through the history of open source at Netflix. He told an anecdote about how one of the developers asked what the policy was for open sourcing a project. The developer was told that the the Netflix policy is to have no policies, just go for it. The developer then asked if he should run it by legal and the reply was “If you think legal review is going to improve your code quality, go ahead!” Over a year later, Netflix has open sourced 18 projects with more on the way.

Adrian then went on talk about the strategy and motivation behind Netflix’s open source offering. There are 4 primary goals.

  1. Establish [Netflix's] solutions as Best Practices / Standards
  2. Hire, Retain and Engage Top Engineers
  3. Build up Netflix Technology Brand
  4. Benefit from a shared ecosystem

Netflix Goals

At a very abstract level, Adrian talked about the single points of failure facing the Netflix platform. He felt that Netflix had achieved a Functional and Scalable platform but they had yet to be Portable. Given some of the high profile outages Netflix has suffered on AWS this is understandable. However, the fact of the matter is that competitors haven’t yet reached the same level of functionality and scalability that AWS has but that day is not far off. So Netflix has chosen to get a head start on portability.

Functional, Scalable, and Portable

Finally, he wrapped things up with some concluding statements on the future of the Netflix platform. These were summed up as:

  1. Functionality and scale now, portability coming
  2. Moving from parts to a platform in 2013
  3. Netflix is fostering an ecosystem
  4. Rapid Evolution – Low MTBIAMSH (Mean Time Between Idea And Making Stuff Happen)

Wrap up

You can find more detail on his talk at the First NetflixOSS Meetup. After the overview was complete, it was time for the lightning talks. In general these were well done and served as a great introduction to some of the open source projects. They also enticed developers to take a deeper dive into the projects during the next section of the meetup where the project leads were offering demos and walk-throughs. That was were I headed next to get more detail on one specific project.

Denominator

Denominator is a Java library for controlling DNS, being built to be as portable as possible, with few dependencies. Adrian Cockroft introduced Denominator as the latest addition to Netflix’s open source stable during his overview.

Denominator

I was particularly interested in this as it’s being led by Adrian Cole, the founder of jclouds. I’ve been working with Cole for the past 6 months on jclouds, ever since it was adopted by Rackspace as our official Java SDK.

After getting a tour of Denominator, I noted a number of similarities in architecture between it and jclouds…naturally. It was interesting to see how he was using Dagger for dependency injection as opposed to the way Guice is used in jclouds. He was also using an interesting tool to power a Denominator command line client. The name escapes me but I’ll update this post once I remember. I’m looking forward to the future of Denominator and its role in making the Netflix platform more portable.

Coda

All in all it was a interesting evening discussing open source software. The content was great and the meetup was very well run. The coup de grâce was the food and drinks provided by Netflix. It might seem like a small or insignificant thing but when a meetup provides an excellent meal beyond the standard fare of pizza and soda, it’s just one more point that sets it above the rest and leaves a lasting impression.

Bon Appétit.

Eats!

 
Leave a comment

Posted by on February 13, 2013 in jclouds, open source

 

Innovation in the jclouds Community

11 Feb

Last week I had the pleasure of hosting the first jclouds meetup of 2013 at the Rackspace offices in San Francisco. The number one thing that struck me during the meetup was the amount of innovation going on and the commitment of the community. We had a great turn out and 4 presentations were given by members. Without further ado, here they are in the order they were presented.

The Rackspace Java SDK Powered by jclouds

What with hosting the meetup and all, I took the opportunity to let the community know about the Rackspace SDKs and how our Java SDK is powered by jclouds. No fork, no separate “distribution”, just committing code, examples, and documentation to the upstream jclouds repos. I kept it short and sweet, followed by a brief demo of jclouds in action working with different OpenStack providers. It also gave me a chance to let everyone know that developer.rackspace.com is now the destination for developers building solutions on the Rackspace Cloud. Here’s my slide deck.

A jclouds High Level Overview

Next up was the founder of jclouds, Adrian Cole, to give everyone an overview and update on where jclouds is and where it’s going. For a lot of the attendees this was their first introduction to jclouds. Here are Adrian’s slides.

This was a real privilege for me as I had yet to meet Adrian person. I’ve been working with him and the jclouds community for that past 6 months but really only knew people by their IRC handles. It was great to meet everyone in person and put a face to those IRC handles.

Pallet and Hadoop

Antoni Batchelli then described how his startup had built Pallet on top of jclouds. Pallet is platform for agile and programmatic automation of infrastructure in the cloud, on server racks or directly on virtual machines. Pallet provides cloud provider and operating system independence, and allows for an unprecedented level of customization. Antoni’s goal is to deliver people from the “Amazon tax” by allowing them them to run their Hadoop jobs on any provider supported by jclouds. To find out more about the solution check out PalletOps.com. Here are his slides.

jclouds-chef and Abiquo

Finally, Ignasi Barrera had travelled from Barcelona, Spain to tell us about how his company Abiquo was using jclouds-chef to bootstrap chef deployments in the cloud. jclouds-chef are components to use the Opscode Chef REST API. Abiquo is a purpose built Cloud Management solution that integrates, operates and optimises your existing legacy infrastructure, processes and services into a single, easy to use, unified platform. Put them together and it allows Abiquo users to easily provision and configure software when they need it. Here are Ignasi’s slides.

Coda

I was thoroughly impressed with the innovation and passion demonstrated by the members of the jclouds community at the meetup. Many stuck around after the presentations were finished to chat about the possibilities that jclouds enables. I’m privileged to be part of such a strong open source community and I’m looking forward to the future of jclouds.

Thanks to everyone for taking the time to come out to the meetup and to those who presented. Cheers.

 
Leave a comment

Posted by on February 11, 2013 in jclouds

 

An Annotated GitHub Workflow

25 Jan

tinker toysAfter you’ve worked with any open source project hosted on GitHub for a while, you get used to its particular workflow. Well here’s mine for jclouds but it’s applicable to many projects. This guide will show you how to commit code to a project hosted on GitHub. The guide assumes you’re as least somewhat familiar with git and already have a GitHub account.

Getting the Repo

  1. Go to https://github.com/jclouds/jclouds and click Fork in the top right hand corner.
  2. Back on your machine, clone the repo
    1. git clone git@github.com:rackspace/jclouds.git
    2. cd jclouds
  3. Add a remote to track the master branch of the jclouds repo (and call it “upstream”).
    1. git remote add --track master upstream https://github.com/jclouds/jclouds
  4. Rebase the changes from upstream just to try it out. I include –tags because jclouds manages their patch releases with tags.
    1. git pull --rebase --tags upstream master
  5. Build jclouds for an Eclipse dev env.
    1. mvn clean install eclipse:eclipse -Dmaven.javadoc.skip=true -DdownloadSources=true -DdownloadJavadocs=true -DskipTests

In my case I’m working out of a Rackspace repo so the fork lives at https://github.com/rackspace/jclouds

Before You Write a Line of Code

It’s always a good idea to let the jclouds community know what you’re up to before you write a line of code. Maybe you want to add a new API or Provider so it’s best to give the community a heads up before sending a massive pull request. The best ways to do that are: open an issue in GitHub, send an email to the jclouds-dev mailing list, or hit up #jclouds on IRC on Freenode.

Day to Day Workflow

  1. Starting on my master branch, I always rebase to make sure I’m up-to-date.
    1. git pull --rebase --tags upstream master
    2. git push
  2. Start a new branch for a new feature.
    1. git checkout -b rax-clb-monitors
  3. Go off and write some code.
  4. When you’re done, check to make sure you changed what you thought you changed. Remove and superflous changes.
    1. git status
    2. git diff
  5. Run the tests.
    1. mvn clean install
  6. When it runs clean, add everything in prep for the commit. Double check it’s all there.
    1. git add .
    2. git status
  7. Commit the code to your local branch.
    1. git commit -m "The Health Monitor API for Rackspace Cloud Load Balancers."
  8. After your commit it’s time to see if there have been any changes to jclouds upstream.
    1. git pull --rebase --tags upstream master
  9. Once everything is clean and has been merged, push the code to a remote branch in your forked repo.
    1. git push --set-upstream origin rax-clb-monitors
  10. Time for some GitHub action. Go to your forked repo. In my case it’s
    1. https://github.com/rackspace/jclouds
  11. You should see a convenient little Pull Request button now. Click on it.
    1. Pull Request
  12. You’ll be sent to the pull request screen where you can add additional comments. Click Send pull request when you’re done.
  13. Now you’ll be on the pull request page.
  14. jclouds has a continuous integration service running provided by CloudBees BuildHive. I’ve seen it take anywhere from 10 minutes to 1 hour to run. Wait patiently. Write a blog post or something.
  15. Once the continuous integration is complete, someone from the jclouds community should be in touch to do a code review. You’ll be contacted through the pull request comments.
  16. After the code review, it’s likely you’ll have some changes to make.
    1. Go back and repeat steps 3-8.
  17. Now when everything is ready to go again, you’ll have a number of commits to send back. However, to keep the commit history of upstream jclouds sane, you’ll need to squash those commits down to one commit only. In the example below, two commits were made on top of the original commit for a total of 3 commits. Check the git log to make sure your commits are at the “top”.
    1. git log --pretty=format:"%h %an %s"
    2. git rebase --interactive HEAD~3
  18. This will open up an editor to start an interactive rebase. You’ll want to “pick” your original commit and “squash” the other commits onto it. Save and close your editor as normal.
  19. squash
  20. Now you’ll jump straight into editing your commit message. In this case I opted to just delete the 2nd and 3rd commit messages as they were only fixes and didn’t change the intent of the original commit.
  21. edit commit messages
  22. Everything is clean and squashed now so it’s time to push it back to GitHub. A force push can be dangerous only if there’s more than one person working on a branch. Once you do this your original pull request to upstream jclouds will be automatically updated. Refresh the pull request page if you like. Nothing for you to do except wait for the continuous integration to complete and then go through the code review cycle again.
    1. git push --force
  23. When the code is finally given the green light it will be merged to the upstream master. At this point the pull request page on GitHub will give you the option to delete the brach. Go for it.
  24. delete branch
  25. Almost there! Now it’s time to clean up. Switch back to your local master branch, pull down your branch that was just merged to the upstream master, push it to your remote master, and delete your local branch to keep things spiffy clean.
    1. git checkout master
    2. git pull --rebase --tags upstream master
    3. git push
    4. git branch --delete rax-clb-monitors

Coda

Notice how I merge changes from the upstream jclouds repo a lot (git pull --rebase --tags upstream master) to stay up-to-date. This is just good hygiene. The more you merge the less likely you’ll be to run into some catastrophic problem at the end.

I’d also encourage you to check out a tool like the bash prompt builder, that I wrote about in New Tools for JSON and Git. It will let you know, at a glance, the state of your local git repo. This can prevent many unfortunate “GAH! I was in the wrong branch!” type screw ups.

At some point in the future I’m going to forget the details above but have dire need of them in an emergency fix type situation so this guide is as much for me as anyone else. There are some shortcuts I could have taken in the commands above but mostly I did things the long way around for illustrative purposes. If you do know some good shortcuts, please share them in the comments below.

 
Leave a comment

Posted by on January 25, 2013 in git, jclouds, open source

 

jclouds, Netflix, and appsworld in San Francisco

14 Jan

I’m going to be in San Francisco the first week of February and it’s going to be a packed week talking about and hacking on open source software. In addition to spending some time at the Rackspace San Francisco office, I’ll be doing a mix of hosting/attending meetups and presenting at the appsworld conference in SF.

jclouds

jcloudsThe Rackspace Java SDK is powered by the open source project jclouds. I’ve been working with the jclouds community since day one of my start at Rackspace but I have yet to meet many of the core developers face to face. The majority of the community is based in San Francisco so I decided to take the opportunity to initiate a couple of meetups. We’ll be hosting two separate meetups at the Rackspace office that week. Here are the details.

jclouds meetup
Date: Feb. 4th
Time: 7:00 pm
Location: 620 Folsom Street #100, SF
Audience: Developers who use jclouds to build solutions

jclouds-dev meetup
Date: Feb. 5th
Time: 7:00 pm
Location: 620 Folsom Street #100, SF
Audience: Developers who develop jclouds itself

NetflixOSS

NetflixNetflix has done an amazing job open sourcing many of their components. They’re a group of top notch engineers working to make resilient applications on the cloud. To herald their commitment to open source they’re hosting the first NetflixOSS Open House on Feb 6th. I’m glad the timing worked out for me to be able to attend this (once I find my way out to Los Gatos). It promises to be a fun and interesting evening, right up until they unleash their simian army on us.

appsworld

appsworldappsworld is a conference relatively new to North America and is focused on mobile apps. The whole conference is geared towards developers and includes four free to attend developer workshop tracks and a free exhibition with over 150 exhibitors. Rackspace is going to have an important presence there and will be hosting a series of workshops.

I’ll be presenting the Rackspace SDKs on Friday morning in the Developer Zone. Here are the details.

A Cloud Platform for Apps: The Rackspace SDKs
Date: Feb. 8th
Time: 9:40 am
Location: Moscone Center West, SF
Agenda:

  1. Introduction to the Rackspace Developer Relations Group
  2. Introduction to the SDKs
  3. Benefits of using the SDKs
  4. Interoperability with OpenStack
  5. Overview of the SDKs
  6. Demo of the Rackspace SDK for Java

Coda

It’s going to be a busy week but I’m looking forward to it. It’s a great chance to connect with some developers and hopefully get some feedback on how Rackspace is serving the developer community. If you’re in or around San Francisco for appsworld or any other reason, please check us out and let us know!

 
Leave a comment

Posted by on January 14, 2013 in jclouds, rackspace

 
← Older posts
Newer posts →
 
Follow

Get every new post delivered to your Inbox.