Spinnaker Success Stories

How did you discover Spinnaker?

We first heard about Spinnaker in 2016, when Netflix published a blog post about it, as a successor of Asgard. We didn't quite have the bandwidth at the time to give it a try, but later on, once our deployment process started to become more and more of a problem, a bell rang, and Spinnaker came into the picture.

What was your experience getting started with Spinnaker?

We started using Spinnaker at Adobe in 2017, when we came across a nice starting guide from AWS. It relied on CloudFormation to create the required resources and it took us under an hour to boot up Spinnaker and hook into our account. We quickly fell in love with it. When it came to using it for our production apps, we encountered an issue though. At the time, Spinnaker was packaged for Debian (Ubuntu), but our company was pushing towards using Centos. We spent quite a lot of time repackaging the Spinnaker services as RPMs and making them work, but after a few months we were able to achieve this goal. It wasn’t ideal, because we were quickly left behind with the new Spinnaker versions and services (eg. Kayenta, as the new service for automatic canary analysis, came into its existence afterwards). We were excited to hear that Spinnaker could be deployed to Kubernetes and once it was packaged as a helm chart, we gave it a try. We are now quite happy with the way things are, with Spinnaker deployed to Kubernetes, using a highly available setup.

Where are you today with Spinnaker? What benefits has it provided to your team?

Today, we're using Spinnaker to deploy some of our most critical apps to production. Most of these apps are deployed to AWS EC2 (virtual machines), but since the beginning of 2019 we've been moving and more towards deploying our microservices to Kubernetes, using Spinnaker. Deploying our apps before using Spinnaker was a cumbersome process, which took quite a lot of engineering effort and was error prone. Spinnaker saved us a lot of man hours and brought a more reliable deployment process for our team. Spinnaker caught the attention of more and more teams inside Adobe and we're now onboarding them in a single Spinnaker instance.

What is missing or could be improved in Spinnaker?

One recurring pain point we hear from our colleagues regarding Spinnaker is the complexity of using it. In order to improve the user experience, we'd love to see more investment in the User Interface. The UI can be quite intimidating for people that are new to Spinnaker, especially when trying to deploying to Kubernetes (using the V2 integration). We're hopeful things will get better in this area and we're already starting to see progress.

How did you discover Spinnaker?

I watched a tech talk in 2017 about how Waze was using Spinnaker and Automated Canary Analysis. I'd been following Spinnaker's development ever since. There had been several discussions in the past about Spinnaker, but these never really materialized. Late 2018, when the time was right for Airbnb to consider a new solution for Continuous Delivery, Spinnaker was added to the shortlist.

What was your experience getting started with Spinnaker?

Airbnb's existing deployment tooling (Deployboard), wasn't built with multi-stage releases in mind. While it does a good job at deploying to a specific environment. It didn't allow for orchestration of deployment pipelines. While we built good validation tooling like integration testing frameworks and traffic replay tools in-house, we couldn't integrate them into the deployment process due to the lack of deployment pipelines. As a result, these tools were not living up to their full potential. As Airbnb transitioned into a Service Oriented Architecture, the deployment process became more complex. Teams ended up writing manual run books with many steps to follow to deploy a service. As the complexity increased, so did the number of incidents. Deployment related incidents started having a material effect on our overall website up-time. Spinnaker provides Automated Canary Analysis out of the box. Which is another big lever for us. Rather than extending the tooling we have, we decided to invest in Spinnaker. We can leverage the community and contribute back, rather than building a closed-source in-house deployment tool. Many of the deployment challenges we face are not unique to just Airbnb.

Where are you today with Spinnaker? What benefits has it provided to your team?

Our tech stack consists of many Java and Ruby backend services running on Kubernetes. Our frontend is powered by Node. Airbnb is doing a large push to migrate all of its infrastructure to Kubernetes. We are migrating our monolith to SOA. We are adopting Spinnaker slowly; rather than doing a forced mass-migration, we want to prove it's value-add for Airbnb engineering. We started by deploying Spinnaker with Spinnaker, so that we could get a handle on what it would take to deploy Airbnb services with Spinnaker. After that, we migrated a small subset of non-critical services. We used customer feedback from these very early adopters to drive feature development. Some Airbnb specific tooling like our in-house CI system needed to be integrated with Spinnaker. Once we'd proven that we could deploy these services, we moved towards onboarding the most critical services onto Spinnaker. The goal here is to prevent regressions. This is where we are currently at. We are in the process migrating all of Airbnb's payments stack and most of our core services to Spinnaker. We are already seeing the effectiveness of Spinnaker and ACA. It has already prevented a dozen regressions. In parallel, we continue feature development and are load-testing Spinnaker to ensure that we can handle all of Airbnb's services. We are planning to open up Spinnaker to all services and make it the default deployment tool in 2020.

What is missing or could be improved in Spinnaker?

Spinnaker's User Interface can be hit or miss. Some parts are not as user-friendly. For example, figuring out why ACA failed can take up to 5 mouse-clicks. We are working on improving some parts of this and are hoping to eventually contribute back to the open source community. We are very excited about the work being done around a plugin-based architecture. While Spinnaker itself is already fairly extensible as a result of it being built in Spring-boot, the team believes this will really turn Spinnaker into an extensible platform for Continuous Delivery. In 2020, once our traffic tooling supports this, we will try and leverage some of the more advanced deployment patterns Spinnaker provides like Blue/Green deployments. We are also excited to use Spinnaker as a more general workflow tool to run things like migrations, database schema changes, and many other processes that can be automated with deployment pipelines. We'd also love to integrate ACA with our feature-flagging system (Trebuchet) so that we can safely launch features and ramp up the rollout over time.

How did you discover Spinnaker?

I was actively using Spinnaker in a previous client project.

What was your experience getting started with Spinnaker?

Installing it on VM was a breeze whereas installing it on K8s needed some extra steps and was not clearly documented.

Where are you today with Spinnaker? What benefits has it provided to your team?

Spinnaker is a good CD tool for K8s deployments.

What is missing or could be improved in Spinnaker?

Documentation could be improved.

How did you discover Spinnaker?

We were planning to move our backends from VMs to Kubernetes. kubectl is a CLI that is used to update k8s' metadata and perform deployments. However, when you are an engineering division with 100 engineers, allowing each one to independently perform `kubectl apply` may result in unexpected state of services, especially when you have 50 of those. And giving Jenkins significant permissions to do that also felt wrong. We wanted Jenkins to do nothing more than its CI job and were looking for something that does CD reliably. We explored Jenkins-X, Harness, Kubeflow and Spinnaker. We tried them all and finally settled with Spinnaker for multiple reasons: 1. Easy pipelines 2. Auth and Authz 3. Capable of maintaining multiple k8s accounts easily: stage, alpha, prod, backup, integration. 4. versioned config maps and secrets. 5. Easy rollbacks... Just to name a few.

What was your experience getting started with Spinnaker?

Since the initial k8s setup was going to be small, we deployed Spinnaker as a part of the cluster itself. As of today, we're migrating Spinnaker to its own cluster because it needs to manage many clusters, so we see having it as a separate entity useful. We had Jenkins create Docker images for applications, and push it to ECR. Jenkins also updated the helm charts if required and pushed it to a chart museum. Spinnaker pipelines take Jenkins as a trigger, and uses the build.properties from Jenkins if required. It then bakes data from build.properties into the helm chart and deploys it to the respective cluster. We used halyard to deploy Spinnaker. We have a set of commands that are part of the values.yaml for Spinnaker. Using helm install deploys Spinnaker onto the prod cluster.

Where are you today with Spinnaker? What benefits has it provided to your team?

Currently Spinnaker serves 50% of all our service covering 70% of all our deployments pipelines. It allows for teams to test their changes on alpha before deploying to prod via manual judgement, which is the most popular feature here. It has reduced the amount of bad code entering production significantly. Moreover, having multiple environments has become easier because deploying in them is easy. Earlier we only had stage and prod as that alone was hard to manage for a small team. The deployment time is significantly lesser than what it was when we were on VMs, almost by 80%. Rollbacks is super easy, esp with versioned configmaps and secrets. This was one thing that went as a feature request in a meeting with Jenkins-X team.

What is missing or could be improved in Spinnaker?

It takes 2 mins for new deployments to show up after a bad deployment has been detected. So it's not possible to immediately rollback on the first error occurrence. Instant rollback trigger is a much required feature. For 2 mins, it wont show the new deployment under infrastructure, hence rollback is not possible.

How did you discover Spinnaker?

After a lot of R&D looking at improving our delivery model, we found Spinnaker to be the best tool for the job.

What was your experience getting started with Spinnaker?

Some software solutions get harder to use the more you find their limitations. Spinnaker is the opposite - we have got faster and faster the more we use it.

Where are you today with Spinnaker? What benefits has it provided to your team?

A few metrics: 1600% improvement in Cycle time in some spaces (68 days to 4 days). 2000% faster to create new services (160 elapsed hours to under a day)

What is missing or could be improved in Spinnaker?

The base Spinnaker docs are not good. Armory provides much more useful documentation.

How did you discover Spinnaker?

We were trying to build an automated Continuous Delivery pipeline and searching for appropriate tools.

What was your experience getting started with Spinnaker?

Spinnaker is integrated with our internal systems, also Jenkins, Gitlab CI. Spinnaker became a core tool for Kubernetes deployments.

Where are you today with Spinnaker? What benefits has it provided to your team?

4 Spinnaker instances. Responsible for the deployment of more than 30 different services and 100s to come.

What is missing or could be improved in Spinnaker?

Lack of documentation related to API and pipeline as a code.

How did you discover Spinnaker?

We had heard of Spinnaker to keep track of Netflix's open source libraries. When we wanted to automate the continuous deployment process, it was the first tool that came to our mind. Because it is a feature proof tool, it has a large community, and we chose Spinnaker because of its platfrom agnostic nature. Our platform consists of ECS and EKS

What was your experience getting started with Spinnaker?

To be honest it was pretty hard. It's a very complex tool. It takes effort to understand the microservice architecture. In our first attempts, we benefited a lot from the slack community and armory documents.

Where are you today with Spinnaker? What benefits has it provided to your team?

We currently have 400+ applications and 2600+ pipelines. Our entire continuous deployment process runs through Spinnaker. We recently switched from halyard to kleat and automate the deployment of the Spinnaker. We deploy Spinnaker via source control. We also generate our pipelines with code. We create all our pipelines automatically with Pipeline as code.

What is missing or could be improved in Spinnaker?

The absence of built-in pipeline as code, we have to parsing json. For large environments, The cloud driver cannot manage memory consumption. Troubleshooting can be made a lot easier. The plugin SDK doesn't support different languages.

How did you discover Spinnaker?

When researching how we were planning on deploying a new set of microservices into production, we happened upon Spinnaker.

What was your experience getting started with Spinnaker?

It took a little bit of time to figure out how to properly configure both Spinnaker itself as well as how we structure our pipelines. This was pre v2 of the Kubernetes provider, so there was a bit of magic happening that was not well understood right away. Once we got a handle on all the features we were able to easily scale out to new services quickly and easily.

Where are you today with Spinnaker? What benefits has it provided to your team?

Today we are deploying a number of different production workloads with several teams across many different EKS clusters. Some teams are under SOC2 controls, while others have not made it to that point as of yet. With this in mind, the pipelines allow for easy auditing and ensuring all changes to pipelines are run through our controls for proper change control management. As we look toward the future we are planning to migrate our largest application to utilize Spinnaker to do all of the heavy lifting work of application management per customer.

What is missing or could be improved in Spinnaker?

Ops is probably the biggest issue we have today. There is not a great way of maintaining Spinnaker while allowing for easily creating a testing environment. The other issue we face as we continue to add clusters to our purview is being able to template out a pipeline to run against every cluster added to Spinnaker. Currently this is a manual operation on all of the infrastructure pieces that our team owns (logging, monitoring, node draining, calico, ingress) to add a new cluster to all of these individual pipelines. A way of templating that out would be beneficial from the operational aspect of managing all of these clusters for the applications that are the same.

Why did you choose Spinnaker?

Early on in Plecto’s journey it was evident that we were growing out of Heroku in terms of the cost. We wanted to move to AWS, but still wanted the ease of automated builds and deployments that Heroku offered. That led us to Asgaard, which was also developed at Netflix, but later deprecated in favor of Spinnaker. So Spinnaker was the logical choice for us.

How is Spinnaker being used at your company today?

All of our production and staging workloads are managed and deployed using Spinnaker. We use Pipelines to enable easy one click, zero downtime deployments for our engineers.

What benefits has it provided to your team?

It means that anyone in the engineering team can safely deploy without having to worry about what’s happening behind the scenes. It also means we can easily manage auto scaling of our services so we don’t have to over provision resources, since the load varies a lot throughout the day.

Why did you choose Spinnaker?

We chose Spinnaker as our Continuous Delivery Platform because it allowed us to provide a consistent Continuous Delivery experience on top of otherwise fragmented infrastructure and workloads. Quantum Metric has grown quickly and not everything we have is running on Kubernetes yet. Spinnaker allows us to tap into a common workflow for promoting artifacts and automated canary analysis regardless of the infrastructure it is running on.

How is Spinnaker being used at your company today?

Spinnaker has managed thousands of deployments across several of our teams as well as provides the orchestration for our ephemeral environments. We are working towards standardizing the configuration of applications in order to lower complexity and increase adoption across more teams and workloads.

What benefits has it provided to your team?

Spinnaker has allowed us to provide a seamless Continuous Delivery experience by abstracting away the complexity of releasing and validating changes to our product. Automated Canary Analysis is being used in several critical and frequently deployed services and we are continuing to drive adoption across other teams through standardized tooling.

How did you discover Spinnaker?

SAP CX Site Reliability Engineering team discovered Spinnaker a year ago. At that time our team was tasked to build a blueprint for a standard CI/CD pipeline that could improve developer's productivity and reduce the lengthy development cycles. We explored various solutions from GitLab, Bamboo, Jenkins and Concourse. During our POCs, we got a lot of success with Spinnaker.

What was your experience getting started with Spinnaker?

The experience in general was a mix of challenges and positiveness coming from the fact that Spinnaker was open source. We spent a lot of time on the various community Slack channels, asking questions and getting help from peers. We also did a lot of trial and error while configuring Spinnaker. But eventually, we started to understand how Spinnaker was architected and it made our experience more intuitive. We did not attend a class or webinar on Spinnaker back then. The decision to use it was solely based on the successful POC we did.

Where are you today with Spinnaker? What benefits has it provided to your team?

Today, we are using Spinnaker to run deployment pipelines for 10 development units. It translates to 30 Kubernetes clusters in production and 40 distinct deployment pipelines based on official templates that our team is maintaining. Our largest development projects are deploying 25 times in production (daily average). We have teams performing 200 daily deployments in what we call PR namespaces, which are temporary namespaces to validate features before they reach the master branch and hit production. Spinnaker has helped us scale our CI/CD service beyond our line of business. We are now offering it as a shared service across SAP. In addition to the scale it provided, the speed to onboard new teams has been another important benefit. By leveraging the Halyard configuration coupled with a GitOps approach, we are able to onboard teams using a self service model. Spinnaker also allows transparency and collaboration, its integration with Slack which was already widely used helps the teams providing insights on their deployment flows.

What is missing or could be improved in Spinnaker?

Documentation and API to manage the Halyard configuration are two important things for us. We would to programmatically configure Spinnaker without having to manipulate config files. our team has started wrapping Halyard into an API, but that work is moving slowly since we want to see where Spinnaker is heading to. We also need more security, especially around the docker images that are used by Spinnaker.

How did you discover Spinnaker?

We love the simple command-line interface for administration, integration with multiple platforms, and easy configurability using pipelines.

What was your experience getting started with Spinnaker?

In our infrastructure, we were using Ansible and slowly Spinnaker started replacing it.

Where are you today with Spinnaker? What benefits has it provided to your team?

We are using Spinnaker in dev, test, and production. We have implemented Spinnaker as a deployment tool for many applications. Key advantages include faster deployments, user confirmation when needed, and access control.

What is missing or could be improved in Spinnaker?

The UI can be made more intuitive and troubleshooting can be made a lot easier.

How did you discover Spinnaker?

I learnt about Spinnaker when I attended Google Cloud Summit in New York City in 2017. I was fascinated by the potential of this technology. It's a true enabler for companies to embrace continuous deployment and change the way software is shipped to production.

What was your experience getting started with Spinnaker?

Prior to using Spinnaker our build/deployment model was built around Jenkins. We used an internally managed Jenkins version to build and ship code to production. Jenkins quickly became a bottleneck with a high price tag for a young startup like Veamly. As part of migrating our CI/CD infrastructure, we ended up using Google Cloud Build and Spinnaker to deploy docker images to our Kubernetes clusters. After we migrated our build process to use Google Cloud Build, we decided to use Spinnaker especially given that we were looking at doing canary releases instead of blue/green deployments. The transition was smooth and done in the course of a day of work.

Where are you today with Spinnaker? What benefits has it provided to your team?

Today, all our production deployments are done using Spinnaker, By doing so, we gained more stability, more flexibility to do rollbacks, and faster deployments. Spinnaker has also offered us more visibility about which Kubernetes cluster will be updated by a given Spinnaker pipeline.

What is missing or could be improved in Spinnaker?

Given our size, we have not yet run into any limitations with Spinnaker.

How did you discover Spinnaker?

I first discovered Spinnaker through the Netflix Engineering tech blog, which I was already following for Java-related topics.

What was your experience getting started with Spinnaker?

It was kind of hard to get started with it back in the old days (circa 2015-2016), then came along halyard, which allowed a smoother experience. But, it still feels a little overwhelming for newcomers to the Spinnaker platform, so I would say better than before but still far from Zero To Hero in a few commands. We should provide a simple halyard template file for the most common installations options (including setting up the provider, authentication, CI tool, etc.) in just one command. More defaults should already be configured, but just in a disabled state.

Where are you today with Spinnaker? What benefits has it provided to your team?

Spinnaker has provided my client (I work for a consulting company) with more stability in their pipelines, speedier deliveries, and a strong understanding of the application delivery process.

What is missing or could be improved in Spinnaker?

My main concern or the missing link has always been a community-standardised way of doing pipeline as code with Spinnaker (PAC). This is something that has prevented Spinnaker adoption in some organisations.