#GlueCon 2014 Notes: Salter: Using Go to provision Saltstack clusters on AWS – Dave Smith, Orchestrate

(Notes by @tomkane)

Salter: Using Go to provision Saltstack clusters on AWS – Dave Smith, Orchestrate

  • Using fanout.io for PubSub.
  • Love Salt. SaltCloud didn’t exist yet, so…
  • History
    • Gen0: Ruby (agh!)
    • Gen1: Python (better…)
    • Gen2: Golang app (best!)
      • Benefit: Single binary
  • What’s it do?
    • Construct reproducible cluster from spec
    • Bootstrap salt master + minions into cluster
    • Provide commands to manage cluster
      • e.g. SSH into all nodes in a cluster (!)
  • Notable features
    • Multithreaded instance initiation
      • Easy to express parallelism in Go
      • e.g. 11 mins to spin up 28 nodes w/ Hadoop, Elastic Search, Hbase, API nodes
    • Sandboxing of creds based on AWS key
      • Also can e.g. change terminal colors when in production (Yes!)
    • Clean logging of highstate for multiple nodes
      • Highstate can be very powerful/dangerous
      • Need insight into the process
      • Addresses some reporting concerns w/ Salt
  • Cluster Spec
    • e.g. https://github.com/orchestrate-io/salter/blob/master/example.cfg
  • Q: What about when a node dies? A: Requires manual intervention. Not auto-scaling.
  • Golang Joy
    • Static compilation / easy distribution
      • “This makes Go the tool to beat for cmd line tools in coming years”
    • Parallelism constructs
      • Essential for this use case. 1 Go routine for each server being spun up
    • Fun, new lang w/ good lib support for AWS & SSH
  • Golang Pain
    • Source/library mgmt
      • Forking a lib on GitHub requires code changes (?!)
    • Typing system can be awkward
      • for-each construct requires reflection
  • Nifty Utility Features
    • ssh – never lookup an IP again
    • hosts – get an easy hosts file for whole cluster
    • info – see internal/external IPs
    • csshx – access all nodes of a certain type