Custom Settings

Furthermore, some users will be more comfortable forgoing Halyard’s configuration generation, or will want to port their old Spinnaker configuration into Halyard. For these users & use-cases, Halyard supports custom Profiles , and custom Service Settings .

Custom Profiles versus Service settings

The two most common ways of customizing your Spinnaker installation is through the use of either a custom profile, for example clouddriver-local.yml, or a service setting, for example clouddriver.yml. Each of these is created in separate subfolders inside the main Halyard deployment profile folder. In a default Halyard profile, this means profiles go in .hal/default/profiles/ while service settings go in .hal/default/service-settings.

Custom profiles are configuration files that override the default configuration files for each Spinnaker Service. Service-Settings are used by Halyard to determine how to build the Spinnaker deployment and contain information specific to how each Spinnaker Service will be deployed.

For example, if you want to substitute which image file will be used for a Spinnaker service, or what environment variables the service will be run with, then you’ll need to modify the service settings for that service.

To override the debugging level specified in clouddriver.yml, create a custom clouddriver-local.yml file and put it in the /profiles/ directory.

Custom Profiles

If you want to supply custom Profiles, for a particular Deployment ($DEPLOYMENT, typically default) place them in ~/.hal/$DEPLOYMENT/profiles/. The will override any profiles generated by Halyard, and will be copied over by Profile name.

For example, if you want to supply a different clouddriver.yml, you can do so by placing it in ~/.hal/$DEPLOYMENT/profiles/clouddriver.yml, and running hal deploy apply.

For any of the Spring based services, any profile matching ${SERVICENAME}-.*\.yml will be copied and applied to $SERVICENAME. Since this approach will completely overwrite the generated service, and you may want to only supply a few flags, you can instead provide them in ${SERVICENAME}-local.yml, which will always be loaded by $SERVICENAME with no further configuration needed. Further Spring Profiles can be activated by configuring their env to include SPRING_PROFILES_ACTIVE entries for your desired service. See the below Custom Service Settings for more information.

You can validate that your Custom Profiles have been picked up and applied to their services by checking ~/.hal/$DEPLOYMENT/history/service-profiles.yml.

Custom Profile for Deck

To supply custom settings or new values that override generated settings for the Deck UI, you can place them in a file called settings-local.js. For example, to enable the chaosMonkey feature flag, you can create the following file:

~/.hal/default/profiles/settings-local.js:

window.spinnakerSettings.feature.chaosMonkey = true

Custom Service Settings

For each service, $SERVICENAME, you can supply a file ~/.hal/$DEPLOYMENT/service-settings/$SERVICENAME.yml to override any settings as found in ~/.hal/$DEPLOYMENT/history/service-settings.yml. For example, if you wanted Echo to listen on port 443 with the JAVA_OPTS="-Xms256m -Xmx512m" environment variables set, you create the following file:

~/.hal/default/service-settings/echo.yml:

port: 443
env:
  JAVA_OPTS: '-Xms256m -Xmx512m'

If you wanted to specify a secure connection to redis, you would create the following file:

~/.hal/default/service-settings/redis.yml:

overrideBaseUrl: redis://someuser:somepassword@localhost:6379

Tweakable Service Settings

Below are all the Service Settings you can configure. Keep in mind the ones generated by Halyard are generally best and are all that we test, so there is no guarantee making changes to these will result in a working Spinnaker cluster.

Kubernetes

Inside the kubernetes item at the top level you can add a number of custom service settings that impact how the Spinnaker services are deployed onto a Kubernetes cluster. This lets you partially configure how the Kubernetes Deployment, Service, Pods and other Kubernetes resources are deployed by Halyard. This allows you to do things like add a Kubernetes secret or emptydir to be mounted as a Volume and VolumeMount, change the default Kubernetes deployment strategy and more.

At a high level, the configurable items inside the Kubernetes service settings are:

imagePullSecrets

If making use of custom artifactIds from an authenticated docker registry imagePullSecrets must be made available in the replicaset definition. To specify imagePullSecrets to custom artifactIds they can be specified as follows:

kubernetes:
  imagePullSecrets:
  - desired-image-pull-secret1
  - desired-image-pull-secret2

podAnnotations, podLabels and serviceAnnotations, serviceLabels

Annotations and labels hold metadata that can be used to specify special behavior within Kubernetes. To apply annotations or labels to the Pod and Service resources for a particular Spinnaker service you can use something like:

kubernetes:
  podAnnotations:
    example/podAnnotation: spinnaker.io
    example/podAnnotation-2: halyard
  podLabels:
    examplePodLabel: examplePodLabelValue
  serviceLabels:
    exampleServiceLabel: exampleServiceLabelValue
  serviceAnnotations:
    example/serviceAnnotation: spinnaker.io
    example/serviceAnnotation-2: halyard

These are all optional ways to control what metadata elements exist on the resources deployed by Halyard. For additional information about how these can be used, you can look at the Annotations or Labels and Selectors pages from Kubernetes.

nodeSelectors

Node selector annotations will put out nodeSelector values in the Pod specification. They generally follow a similar syntax to the annotations and labels. For example:

kubernetes:
  nodeSelector:
     exampleNodeKey: exampleNodeValue
  deploymentEnvironment:
    nodeSelector:
      exampleNodeKey: exampleNodeValue

Additional information regarding Node Selectors can be found in the Kubernetes NodeSelector documentation .

affinity

Affinity annotations generate affinity values in the Pod specification and are more expressive than nodeSelector. You can set precedence rather than specify an affinity hard requirement. The two types are node affinity and inter-pod affinity/anti-affinity.

Provide configuration per spinnaker component like the below example.

kubernetes:
  deploymentEnvironment:
    affinity:
      spin-clouddriver:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: spinnaker
                operator: In
                values:
                - 'true'

See the Kubernetes affinity documentation for more information.

useExecHealthCheck

By default halyard deploys services with an exec-based health check in order to improve compatibility with Istio. This however can break functionality for implementations of Load Balancer service types and Ingress Controllers that rely on having an http health check to validate. Setting kubernetes.useExecHealthCheck: false will switch the check method to be http based for such use cases.

Using Custom Volumes

You can define a set of custom volumes for a particular service that will become Volume and VolumeMount items in the generated pod specifications when Halyard deploys Spinnaker on Kubernetes. This can be useful if you need to add custom CA Certificates or other files to Spinnaker service consistently.

The Kubernetes service settings allow you to mount volumes of type configMap, secret and emptyDir with the below syntax.

kubernetes:
  volumes:
  - id: secret-example-vol
    type: secret
    mountPath: /path/to/location/of/secret
  - id: configmap-example-vol
    type: configMap
    mountPath: /path/to/location/of/configMap
  - id: emptydir-example-vol
    type: emptyDir
    mountPath: /path/to/location/of/emptyDir

Make sure that the id field matches the name of the existing secret or configmap resource in Kubernetes that you need mounted.

Deployment Strategy

You can change the deployment strategy that Kubernetes uses when deploying Spinnaker services. For example, if you wanted to adjust the maxSurge and maxUnavailable percentages for a rolling update of Clouddriver, you would define a clouddriver.yml file in your Halyard service-settings like the one below.

kubernetes:
  deploymentStrategy:
    type: rollingUpdate
    strategySettings:
      maxSurge: 100%
      maxUnavailable: 50%

If you wanted to use a Recreate strategy instead, you would use:

kubernetes:
  deploymentStrategy:
    type: recreate

For additional information regarding the implication of each of these deployment strategies, consult the Updating a Deployment documentation.