#GlueCon 2014 Notes: Salter: Using Go to provision Saltstack clusters on AWS – Dave Smith, Orchestrate

(Notes by @tomkane)

Salter: Using Go to provision Saltstack clusters on AWS – Dave Smith, Orchestrate

• Using fanout.io for PubSub.
• Love Salt. SaltCloud didn’t exist yet, so…
• History
  • Gen0: Ruby (agh!)
  • Gen1: Python (better…)
  • Gen2: Golang app (best!)
    • Benefit: Single binary
• What’s it do?
  • Construct reproducible cluster from spec
  • Bootstrap salt master + minions into cluster
  • Provide commands to manage cluster
    • e.g. SSH into all nodes in a cluster (!)
• Notable features
  • Multithreaded instance initiation
    • Easy to express parallelism in Go
    • e.g. 11 mins to spin up 28 nodes w/ Hadoop, Elastic Search, Hbase, API nodes
  • Sandboxing of creds based on AWS key
    • Also can e.g. change terminal colors when in production (Yes!)
  • Clean logging of highstate for multiple nodes
    • Highstate can be very powerful/dangerous
    • Need insight into the process
    • Addresses some reporting concerns w/ Salt
• Cluster Spec
  • e.g. https://github.com/orchestrate-io/salter/blob/master/example.cfg
• Q: What about when a node dies? A: Requires manual intervention. Not auto-scaling.
• Golang Joy
  • Static compilation / easy distribution
    • “This makes Go the tool to beat for cmd line tools in coming years”
  • Parallelism constructs
    • Essential for this use case. 1 Go routine for each server being spun up
  • Fun, new lang w/ good lib support for AWS & SSH
• Golang Pain
  • Source/library mgmt
    • Forking a lib on GitHub requires code changes (?!)
  • Typing system can be awkward
    • for-each construct requires reflection
• Nifty Utility Features
  • ssh – never lookup an IP again
  • hosts – get an easy hosts file for whole cluster
  • info – see internal/external IPs
  • csshx – access all nodes of a certain type