Public Continuous Integration Builds for our OSS Projects

by Gareth Bowles

As Netflix continues to open source our platform (26 projects at the last count), our GitHub repos are seeing more and more changes, both commits from the project owners and pull requests from external contributors.

To provide more visibility into the quality of those changes, we're setting up public builds for all our open source projects so that everyone can see the latest build status and test results.

  

The builds run on a public Jenkins hosted by our friends at CloudBees, using their Dev@Cloud service which is free for open source projects.

Building Commits to Master

We build and test each commit to a project's master branch.  The current build status is shown in the README on the GitHub page.

Clicking on the build status badge takes us to the Jenkins build that gets triggered on every push to the master, where we can look at the build details and test results.

Verifying Pull Requests

We automatically verify pull requests by merging the contents of the pull request with the current tip of the branch that the request was made against, then building and running tests.  The pull request builds are executed in an isolated environment to protect against malicious code.  

When the build finishes, it will add a comment to the pull request in GitHub. 

Auto-Generating Build Jobs

We use the Jenkins Job DSL plugin, available via the public Jenkins plugin repository, to automate the creation of master and pull request builds for each new OSS project.  The plugin allows Jenkins jobs to be created by writing a small amount of Groovy code; since the jobs for each project are very similar (the only significant variation is the GitHub repo URL), this is much faster than clicking through the UI and allows us to have a one-click build setup for new projects.  Here's the Groovy DSL code that sets up the master build job; it's available on GitHub at https://github.com/Netflix-Skunkworks/netflixoss-dsl-seed.

   

    job {

        name "${repo.name}-master"

        description ellipsize(repo.description, 255)

        logRotator(60,-1,-1,20)

        scm {

            git(repo.git_url, 'master')

        }

        jdk('Sun JDK 1.6 (latest)')

        configure { project ->

            project / triggers / 'com.cloudbees.jenkins.GitHubPushTrigger'(plugin:'github@1.5') / spec {

            }

        }

        steps {

            gradle('clean build')

        }

        publishers {

            archiveJunit('**/build/test-results/TEST*.xml')

        }

    }

If you would like to contribute to any of our open source projects, please take a look at http://netflix.github.com. You can follow us on twitter at @NetflixOSS.

We will be hosting a NetflixOSS Open House on the 13th of March, 2013 (limited seats. please RSVP), during which we will showcase our public CI builds and other OSS projects.

We are constantly looking for great talent to join us and we welcome you to take a look at our Jobs page or contact @bettyatnetflix for positions in the Engineering Tools team.