We launched the Netflix Open Source Software Cloud Prize in March 2013 and it got a lot of attention in the press and blogosphere. Six months later we closed the contest, took a good look at the entrants, picked the best as nominees and the panel of distinguished judges decided the winners in each category, The final winners were announced during Werner Vogel’s keynote at AWS Re:Invent in on November 14th 2013.
The ten winners all put in a lot of work to earn their prizes. They each won a trip to Las Vegas and a ticket for AWS Re:Invent, a Cloud Monkey trophy, $10,000 prize money from Netflix and $5000 in AWS credits from Amazon. After the keynote was over we all went on stage to get our photo taken with Werner Vogels.
Peter Sankauskas (@pas256) is a software engineer living in Silicon Valley, and the founder of Answers for AWS (@Answers4AWS). He specializes in automation, scaling and Amazon Web Services. Peter contributed Ansible playbooks, Cloud Formation templates and many pre-built AMIs to make it easier for everyone else to get started with NetflixOSS, and put Asgard and Edda in the AWS Marketplace. He recently started his own company AnsWerS to help people who want to move to AWS. Getting started with NetflixOSS is made harder because there are 35 different projects to figure out. Peter has created an extremely useful and simple on-ramp for new NetflixOSS users.
Chris Grzegorczyk (eucaflix, grze, of Goleta California) is Chief Architect and Co-Founder, and Vic Iglesias is Quality and Release Manager at Eucalyptus Systems. Eucalyptus have been using NetflixOSS to provide a proof point for portability of applications from AWS to private clouds based on Eucalyptus. Eucalyptus is open source software for building private and hybrid clouds that are compatible with AWS APIs. Their submission enables NetflixOSS projects to treat Eucalyptus as an additional AWS region and to deploy applications to AWS regions and Eucalyptus datacenters from the same Asgard console.
In June 2013 they shipped a major update to Eucalyptus that included advanced AWS features such as Autoscale Groups that NetflixOSS depends on. Eucalyptus have demonstrated working code at several Netflix meetups and have really helped promote the NetflixOSS ecosystem.
IBM had previously created a demonstration application called Acme Air for their Websphere tools running on IBM Smartcloud. It was a fairly conventional enterprise architecture application, with a Java front end and a database back end. For their winning prize entry, Andrew Spyker (aspyker, of Raleigh North Carolina) figured out how to re-implement Acme Air as a cloud native example application using NetflixOSS libraries and component services, running on AWS. He then ran some benchmark stress tests to demonstrate scalability. This was demonstrated at a Netflix Meetup last summer. The Acme Air example application combines several NetflixOSS projects. The Eureka service registry, Hystrix circuit breaker pattern, Karyon base server framework, Ribbon http client and Asgard provisioning portal. IBM used NetflixOSS to get a deeper understanding of Cloud Native architecture and tools, which it can apply to helping enterprise customers make the transition to cloud.
The Reactive Extensions (Rx) pattern is one of the most advanced and powerful concepts for structuring code to come out in recent years. The original work on Rx at Microsoft by Eric Meijer inspired Netflix to create the RxJava project. We started with a subset of Rx functionality and left a lot of “to do” areas. As the project matured we began to extend RxJava to include other JVM based languages and Joachim Hofer (jmhofer, Möhrendorf, Germany) has made major contribution to type safety and Scala support, with over thirty pull requests.
Joachim works at Imbus AG, Möhrendorf, Germany, he’s lead developer of an agile product team and a Scala enthusiast working on moving their stack from J2EE to Scala/Play/Akka/Spray/RxJava.
Anyone familiar with Hadoop tools and the big data ecosystem knows about the Pig language. It provides a way to specify a high level dataflow for processing but the Pig scripts can get complex and hard to debug. Netflix built and open sourced a visualization and monitoring tool called Lipstick, and it was adopted by Mark Roddy at a vendor called Mortar (mortardata, of New York, NY) who worked with us to generalize some of the interfaces and integrate it with their own Pig based Hadoop platform. We saved Mortar from having to create their own tool to do this, and Netflix now has an enthusiastic partner to help to improve and extend Lipstick so everyone who uses it benefits.
Jakub Narloch (jmnarloch, of Szczecin, Poland) created a test suite for NetflixOSS Karyon based on JBoss Arquillian. The extension integrated with Karyon/Google Guice dependency injection functionality allowing to write tests that directly access the application auto scanned components. The tests are executed in the application container. Arquillian brings wide support for different containers including Tomcat, Jetty and JBoss AS. Karyon is the base server that underpins NetflixOSS services and acts as the starting point for developing new services. Since Genie is based on Karyon, we were able to leverage this integration to use Arquilian to test Genie, and the changes have been merged into the code that Netflix uses internally.
Jakub Narloch is a software engineer working at Samsung Electronics. He received the JBoss Community Recognition Award this year for his open source contributions. In the past year he has been actively helping to develop the JBoss Arquillian project, authoring four completely new extensions and helping to shape many others. His adventure with the open source world began a couple of years earlier and he has also contributed code to projects like Spring Framework, Castor XML and NetflixOSS. Last year he graduated with honors Computer Science from Warsaw University of Technology with an MSc degree. In the past he took part in two editions of Google Summer of Code and in his free time he likes to solve the software development contests held by TopCoder Inc.
In the real world complex web service APIs are hard to manage, and NetflixOSS includes the Zuul API gateway, which is used to authenticate process and route http requests. The next winner is Neil Beveridge (neilbeveridge, of Kent, United Kingdom). He was interested in porting the Zuul container from Tomcat to Netty, which also provides non-blocking output requests, and benchmarking the difference. Neil ran the benchmarks with help from Raamnath Mani, Fanta Gizaw and Will Tomlin at Hotels.com. They ran into an interesting problem with Netty consuming excess CPU and running slower than the original Tomcat version, and then ran into the contest deadline, but have since continued work to debug and tune the Netty code and come up with higher performance for Netty and some comparisons of cloud and bare metal performance for Zuul. Since Netflix is also looking at moving some of our services from Tomcat to Netty, this is a useful and timely contribution. It’s also helpful to other people considering using Zuul to have some published benchmarks to show the throughput on common AWS instance types.
Although the primary storage used by Netflix is based on Cassandra, we also use AWS RDS to create many small MySQL databases for specific purposes. Other AWS customers use RDS much more heavily. Jiaqi Guo (jiaqi, Chicago, Illinois) has built Datamung to automate backup of RDS to S3 and replication of backups across regions for disaster recovery. Datamung is a web-based, Simple Workflow driven application that backs up RDS MySQL databases into S3 objects by launching an EC2 instance and running the mysqldump command. It makes it possible to replicate RDS across regions, VPC, accounts or outside the AWS network.
When we started to build the Denominator library for portable DNS management we contacted Neustar to discuss their UltraDNS product, and made contact with Jeff Damick (jdamick, of South Riding, Virginia). His input as we structured the early versions of Denominator was extremely useful, and provides a great example of the power of developing code in public. We were able to tap into his years of experience with DNS management, and he was able to contribute code, tests and fixes to the Denominator code and fixes to the UltraDNS API itself.
Justin Santa Barbara (justinsb of San Franciso, California) decided to make the Chaos Monkey far more evil, and created fourteen new variants, a “barrel of chaos monkeys”. They interfere with the network, causing routing failure, packet loss, network data corruption and extra network latency. They block access to DNS, S3, DynamoDB and the EC2 control plane. They interfere with storage, by disconnecting EBS volumes, filling up the root disk, and saturating the disks with IO requests. They interfere with the CPU by consuming all the spare cycles, or killing off all the processes written in Python or Java. When run, a random selection is made, and the victims suffer the consequences. This is an excellent but scary workout for our monitoring and repair/replacement automation.
We are pleased to have such a wide variety of winners, from individuals around the world, small and large companies, vendors and end users. Many thanks to all of them for the work they have put into helping grow the NetflixOSS ecosystem, and thanks to everyone else who just uses NetflixOSS or entered the contest but didn’t make the final cut.
comment 0 التعليقات:
more_vertsentiment_satisfied Emoticon