Orca’s execution state is stored in Redis by default, but can be configured for SQL. In this topology, Redis is still required for the work queue. Using SQL for execution state will make your Spinnaker installation more durable.
This guide will go over MySQL setup, how to configure Orca, as well as how to perform a zero-downtime migration from Redis to SQL.
If you already have an Orca deployment, you should also refer to the Redis to SQL Migration Guide.
MySQL 5.7 Setup
Orca ships with MySQL drivers by default, but you can include your own JDBC drivers on the classpath if you need to connect to a different database.
Orca has been developed and tested targeting MySQL 5.7. As part of this, setting MySQL’s
tx_isolation value to
READ-COMMITTED is essential to successfully running Orca in SQL.
While Orca will attempt to set this on connection sessions, it is better to have it set on the database itself.
The SQL integration is configured to support a
migration user and a
migration user will only be used to perform schema changes on the database, whereas the
service user will be used for runtime traffic.
Before deploying Orca, the schema and database uses must first be manually setup:
- Set MySQL Server variable
READ-COMMITTED. Refer to MySQL Server System Variables.
From the MySQL Server command line run
set tx_isolation = 'READ-COMMITTED';
- Setup the schema and database users
CREATE SCHEMA `orca` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, EXECUTE, SHOW VIEW ON `orca`.* TO 'orca_service'@'%'; -- IDENTIFIED BY "password" if using password based auth GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, LOCK TABLES, EXECUTE, SHOW VIEW ON `orca`.* TO 'orca_migrate'@'%'; -- IDENTIFIED BY "password" if using password based auth
When Orca starts up, it will perform database migrations to ensure its running the correct schema. It is safe to start multiple Orca services at the same time, even if migrations need to be run.
Configuring Orca for SQL
This configuration is your baseline for Orca to talk to SQL, in
sql: enabled: true connectionPools: default: default: true # additional connection pool parameters are available here, # for more detail and to view defaults, see: # https://github.com/spinnaker/kork/blob/master/kork-sql/src/main/kotlin/com/netflix/spinnaker/kork/sql/config/ConnectionPoolProperties.kt jdbcUrl: jdbc:mysql://localhost:3306/orca user: orca_service password: hunter2 # MariaDB-specific: maxPoolSize: 50 migration: jdbcUrl: jdbc:mysql://localhost:3306/orca user: orca_migrate password: hunter2 # Ensure we're only using SQL for accessing execution state executionRepository: sql: enabled: true redis: enabled: false # Reporting on active execution metrics will be handled by SQL monitor: activeExecutions: redis: false # Use SQL for Orca's work queue # Settings from Netflix and may require adjustment for your environment # Only validated with AWS Aurora MySQL 5.7 # Please PR if you have success with other databases keiko: queue: sql: enabled: true redis: enabled: false queue: zombieCheck: enabled: true pendingExecutionService: sql: enabled: true redis: enabled: false
orca.yml overwrites the configuration generated by Halyard.
In case you have deployed Spinnaker using Halyard, you need to add the configuration above to
Read more about profiles and service-settings here.
The default MySQL Connector for Aurora MySQL 5.7 should be fine, but you may also setup Orca to use the MariaDB JDBC driver over MySQL Connector.
The MariaDB driver is Aurora clustering aware, which takes care of automatic master failover operations. Due to licensing issues, Orca cannot ship with the MariaDB driver.
An example of wiring up MariaDB into Orca can be found here: robzienert/orca-mariadb-extension.
Netflix’s Amazon Aurora Example
While vanilla MySQL provides more durability and performance over Redis, Netflix additionally uses Amazon Aurora MySQL 5.7. If you’d like to configure Orca to use Aurora as well, here is how Netflix has it set up.
IMPORTANT: This configuration is for multi-region Aurora replication. If you are only deploying Aurora into a single region, don’t enable any binlog settings.
Aurora Parameter Groups
Aurora DB Cluster Parameter Group