#GlueCon 2014 Notes: Salter: Using Go to provision Saltstack clusters on AWS – Dave Smith, Orchestrate

(Notes by @tomkane) Salter: Using Go to provision Saltstack clusters on AWS – Dave Smith, Orchestrate Using fanout.io for PubSub. Love Salt. SaltCloud didn’t exist yet, so… History Gen0: Ruby (agh!) Gen1: Python (better…) Gen2: Golang app (best!) Benefit: Single binary What’s it do? Construct reproducible cluster from spec Bootstrap salt master + minions into cluster Provide commands to manage cluster e.g. SSH into all nodes in a cluster (!) Notable features Multithreaded instance initiation Easy to express parallelism in Go e.g. 11 mins to spin up 28 nodes w/ Hadoop, Elastic Search, Hbase, API nodes Sandboxing of creds based on AWS key Also can e.g. change terminal colors when in production (Yes!) Clean logging of highstate for multiple nodes Highstate can be very powerful/dangerous Need insight into the process Addresses some reporting concerns w/ Salt Cluster Spec e.g. https://github.com/orchestrate-io/salter/blob/master/example.cfg Q: What about when a node dies? A: Requires manual intervention. Not auto-scaling. Golang Joy Static compilation / easy distribution “This makes Go the tool to beat for cmd line tools in coming years” Parallelism constructs Essential for this use case. 1 Go routine for each server being spun up Fun, new lang w/ good lib support for AWS & SSH Golang Pain Source/library mgmt Forking a lib on GitHub requires code changes (?!) Typing system can be awkward for-each construct requires reflection Nifty Utility Features ssh – never lookup an IP again hosts – get an easy hosts file for whole cluster info – see internal/external IPs csshx – access all nodes of a certain type

James Higginbotham

May 22, 2014 · 2 min read