The inaugural Netflix Open Source Software (NetflixOSS) meetup was held at Netflix headquarters in Los Gatos California on February 6th 2013. We had over 200 attendees; individual developers, academics, vendors and companies, from small startups to the largest global corporations.
For those who missed it, a video of the presentations made at the event is included below. We also had visitors from TechCrunch and GigaOM writing stories about us.
How to host a Tech Meetup
We were very happy with the way the meeting itself worked out, but Paul Guth wrote a wonderful blog post on what he saw. http://constructolution.wordpress.com/2013/02/06/netflix-teaches-everyone-how-to-host-a-tech-meetup/“Netflix has once again set the bar. Not with their technology this time – but with their organizing. I just got back from the first meetup of the NetflixOSS group - and it was spectacular.”
The inside story of how this meetup happened provides another example of how Netflix culture works. We don’t have a recipe for a meetup, there was no master plan with a Steve Jobs like attention to getting the fine details exactly right. There was no process. Instead, we told people the high level goals of what we were trying to do, got out of the way and trusted that they could figure it out or ask for clarification as needed. On the day of the event we had excellent facilities support setting everything up, making sure people knew where to go, and staying up very late to put it all away. We had wonderful asian finger food (sushi!) and plenty of beer and wine. There was signage and large monitors in 10 separate demo stations with the engineers that own individual NetflixOSS projects. Ruslan led the event overall and structured the talks, Adrian worked on the what/why message and how it works slides, Joris from PR got journalists to come along. Betty coordinated the event setup. The Netflix design team came up with a NetflixOSS logo, T-shirt design and TechTattoo stickers. Leslie turned the designs into actual T-shirts and stickers. A lot of work from a lot of people went into the meetup, but it was fun and frictionless, with immediate buy-in. It sounds too good to be true but good ideas “get legs” at Netflix, and take off in a way that isn’t often seen at other companies. Netflix gets out of the way of innovation. We get more done with fewer people in less time, and this is a key ingredient to maintaining high talent density, because employees aren’t being frustrated by bureaucracy, the default behavior is to trust their judgement. Just like with NetflixOSS components, we apply similar philosophy with our people - whole is greater than the sum of the parts. Together, the amazing people that we have are able to accomplish much more, than all of their individual accomplishments put together.
NetflixOSS History
Ruslan opened by showing everyone an email exchange. In mid 2011 Jordan sent an email to some managers asking what the process was to open-source a project, the reply was that there is no process or policy, just go ahead. Jordan then asked if he should just put on Apache license headers and show it to legal, and the response was “If you think legal review is going to improve your code quality, go ahead!”. When the code was ready it was released in late 2011 as Curator. During 2012 another 15 projects were added, and three more already in 2013.We were releasing the platform that runs Netflix streaming one piece at a time, and other people started using bits and pieces individually. The transition we are making in 2013 is that we are putting the puzzle pieces together as a coherent platform, branding it NetflixOSS and will make it easy to adopt it as a complete Platform as a Service solution. NetflixOSS supports teams of developers who are deploying large scale globally distributed applications. Netflix has hundreds of developers deploying on tens of thousands of instances across several AWS Regions to support the global streaming product.
Why Open Source the Platform?
In 2009-2010 Netflix was figuring out the architecture from first principles, trying to follow AWS guidelines and building a platform based on scalabiity experience some of our engineers had gained at places like Yahoo, eBay, and Google. In 2011 and 2012 we started talking about how we had built a large scale cloud-native application, and other companies began to follow our patterns. What was bleeding edge innovation in 2009 became accepted best practices by 2012, and is becoming a commodity in 2013-2014. Many competing cloud platforms have appeared, and by making it easy for people to adopt NetflixOSS we hope to become part of a larger ecosystem rather than having the industry pass us by.When we started Open Sourcing pieces of our infrastructure, we had several goals for the program - outlined in this Techblog post. We’re seeing great adoption and engagement across many developers and companies, and increasing stream of feedback and external contributions. There is a growing number of 3rd party projects that utilize NetflixOSS components, some of them are listed on our Github page.
Putting it all Together
We released NetflixOSS bit by bit, and we don’t have a naming convention, so it can be hard to figure out how they fit together. Adrian presented a series of slides that explained how they fit together to form a build system, a set of services that support the platform, a collection of libraries that applications build against, and testing and maintenance functionality.
The Portability Question
When we built our platform we focused on what mattered to Netflix: scalability so we could roll out our streaming product globally; functionality so that our developers could build faster; and availability by leveraging multiple availability zones in each region. We regard portability as an issue we can defer, since AWS is so far ahead of the market in terms of functionality and scale, and we don’t want to hobble ourselves in the short term. However as we share the NetflixOSS platform, there is demand from end users and vendors to begin porting some of the projects for use in datacenter and hybrid cloud configurations. For the foreseeable future Netflix is planning to stay on AWS, but in the long term portability may be useful to us as well.A New Project - Denominator
We announced a brand new project at the Meetup; we are working on multi-region failover and traffic sharing patterns to provide higher availability for the streaming service during regional outages caused by our own bugs and AWS issues. To do this we need to directly control the DNS configuration that routes users to each region and each zone. When we looked at the features and vendors in this space we found that we were already using AWS Route53, which has a nice API but is missing some advanced features; Neustar UltraDNS, which has a SOAP based API; and DynECT, which has an awkwardly structured REST API. We couldn’t find a Java based API that abstracted all these vendors into the common set of capabilities that we are interested in, so we are creating one. The idea is that any feature that is supported by more than one vendor API is the highest common denominator, and that functionality can be switched between vendors as needed, or in the event of a DNS vendor outage.With most NetflixOSS projects, we are running the code in production at the point where we open source it on github. In the case of Denominator we are already sharing the code with DNS vendors who are helping us get the abstraction model right, and we will make it generally available during the development process. Denominator is a Java library for controlling DNS, we are building it to be as portable as possible, with few dependencies. We will embed it in services such as Edda, which collects the historical state of our cloud. This project is being led by Adrian Cole, who is well known as the author of the cross platform jClouds open source project. He recently joined Netflix and is bringing a lot of valuable experience to our platform team.
comment 0 التعليقات:
more_vertsentiment_satisfied Emoticon