Sid Carter

Getting it on with clojure

About a week or two ago, I decided to focus on two things. One would be SaltStack and the other would be Clojure.

Over the last week, I’ve learnt quite a bit about clojure and there’s lot more to learn. I wanted to document all the stuff I’ve found in the process of learning clojure.

I will add more here, as I continue to learn and explore clojure. Also, the folks on #clojure @Freenode are super friendly and helpful.

I'm back with SaltStack

So, I resurrected my old blog from the git repository, changed the background and and got the site back online. Most everything is still the same, cause why fix something that isn’t broken?

I wanted to use a different framework for my website, but nothing was easy, minimalist or up to my taste.. So, I’m back to using Jekyll and Amazon S3 for my hosting, cause pourquoi pas! It’s cheap and simple. All my templates and updates are synced to a private repository and so, everything is back the way it was. Tadaaaa!!

We use a lot of different software and tools to manage and monitor systems and services at my workplace. I thought it would be a good idea to get back to blogging about all the stuff I do. All of this is stuff I learnt online and some of it I figured out on my own, because there just wasn’t enough information either in the manuals or online.

We use ansible to do a lot of cloud orchestration, deployment of services etc., Lately, I’ve been looking at how we can improve the way we do things and have been trying out SaltStack. I’ve been finding it a lot more interesting. Here’s what I did to get started with it. I used the SaltStack walkthrough as my starting point.

Before you start, ensure you have the EPEL repo installed, since it contains the necessary salt packages. Do the following and you should have the latest version of epel release for your distro:

$ sudo yum install epel-release

I installed the zeromq4 COPR repository, because I wanted to use SaltStack with zeromq4 and not zeromq3, which is the default on CentOS 6. Ensure you download the repository for your specific distro. For CentOS 6, I did the following:

$ cd /etc/yum.repos.d/
$ sudo wget -c https://copr.fedoraproject.org/coprs/saltstack/zeromq4/repo/epel-6/saltstack-zeromq4-epel-6.repo

Ensure the zeromq4 COPR repository is enabled by setting enabled=1 in the repo file you just downloaded. After that, I installed salt using the following command:

$ sudo yum install salt salt-api salt-cloud salt-master salt-minion salt-ssh salt-syndic --enablerepo epel

That automagically picks the latest version of zeromq4 and salt packages. You don’t really need all those packages, but I installed all of them, since I plan to use them as I go. You just need salt, salt-master and salt-minion for a typical install and salt-cloud if you plan to use salt to manage your cloud infrastructure. I plan to use salt-cloud for all the servers we have in HP Cloud and AWS.

I did not change anything in /etc/salt/master, since by default salt-master runs on all interfaces. You can change this line in /etc/salt/master to change the interface IP you want it to bind to:

#interface: 0.0.0.0

There are lots of other changes that need to be done, but I want to start with the basic setup first. Next, I edited /etc/salt/minion to point to the master and identify the minion as follows:

# check with the master
master: saltmaster.domain.com
# say my name
id: minion1

Restart salt-master using sudo /etc/init.d/salt-master restart and if all goes well, restart salt-minion on the same host using sudo /etc/init.d/salt-minion restart.

Once the salt-master and salt-minion restarted successfully, I ran sudo salt-key -L to list all the keys on the master. You should see a list of all accepted, rejected and unaccepted keys. You should see the minion name you set using id in your minion config here. If you don’t see that or you see a different name, you should check your config.

You can then run sudo salt-key -A and accept the keys waiting to be accepted. You’re now ready to do some basic commands on your minion. In our case, the minion is on the mater itself.

If you do a test.ping, you should see a response like this:

$ sudo salt minion1 test.ping
minion1:
    True

or perhaps you want to check the uptime of the host:

$ sudo salt minion1 status.uptime
minion1:
     05:07:37 up 16 days, 23:42,  1 user,  load average: 0.00, 0.00, 0.00

There’s of course a ton of commands that you can run against the minion now. Refer to SaltStack documentation or this cheat sheet to get started.

Next time, we’ll take a look into salt states and salt-cloud.

Things that interest me right now

I’m intrigued by Stripe and Stellar and the things you can do with it. I would like to know more about the regulations around currencies and trading in the US, so I can work on some ideas I have. The ideas themselves are simple, but I want to look at making things cheaper and better for everyone involved.

Amazon AWS is fantastic and so is Azure (but I can’t try it like I can try AWS, so I don’t care much for it right now) and the things you can do with them are astounding.

The technology landscape is vastly different and more energetic than it was two years ago. Everything is being democratized and I like the look and feel of it. Incredible power in the hands of everyone, if you know how to harness it.

The world is a different place and I’m excited by the possibilities.

Carpe diem, baby. Carpe diem!

Scala and SBT

I’m figuring out Scala and in the process learning about SBT to build my code. Leaving this useful page about build file examples here for future reference.

My Scala mechanize code is going pretty well here.

Next little project – write some code to get stock quotes. I should be able to use mechanize without a prob for that.

Scaaaalaaaa!

I’ve created a few scripts for AWS EC2, S3 and and a playbook for creating an instance using Ansible. All the code is available here. In that same repository, there’s a script I found which helps you get all the AWS Usage reports. I might use that to figure out how my usage and billing are correlated.

And while investigating that, I found this repository that’s ported mechanize to Scala. I plan to use that to get the info in my bank, so I can track the changes and perhaps stop using stupid mint, which often complains about connection issues. [update: this repository needs to be fixed. Fixing :D]

Now, I want to get some stuff done with Hadoop or EMR. If I want to do MapReduce stuff, why not use EMR and Scalding?

This time, I get to learn and do it all in Scaaaaaaalaaaaaa!