Hosting a Rails App on Cloud Foundry - first impressions

From time to time I have been known to write a bit of code and whenever one writes a web application, there is always the question of hosting. I've done my time in Ops and I can certainly deploy an application to a VPS and run the surrounding infrastructure to make it work - however, that all sounds like more work than I'm willing to put in. This is the world of Cloud hosting and I'd like to spend my time writing applications, not deployment scripts. What I want is something I can throw code at and have it sort itself out but for my own projects the price needs to be low so I'm not spending a ton of money every month on my own games.

This is an interesting niche as I don't have the same requirements for my own stuff as I would for professional hosting. Initially my requirements were:

• Very low monthly cost
• Rails 5
• Database (probably postgres)
• Ability to hook it into some kind of CI (ideally Codeship, as I'm already using that)

For my own projects I'm not that bothered about high capacity, or extensive DR. These are great, but are also expensive.

I'm going to end up on Heroku, because the free tier appears to do everything I want and more. However along the way I tried out Cloud Foundry so I thought it worth writing up how I got started.

Easy stuff first

I signed up for an account then created an org and a space on the dashboard. I also created a database within the space (no binding - it's better to do that with a manifest). This was all achievable via the web interface. The postgres service has the option of a free database, limited to 20mb storage.

Next, I installed the command line interface and logged in (cf l), choosing the space as the default.

Preparing the application

A Rails 5 application needs no additional configuration, beyond migrating it from sqlite to postgres. The easiest way to tie the application to the production database is via a manifest file. Mine looks like this:

---
applications:
- name: yip-helper
  random-route: true
  memory: 128M
  instances: 1
  path: .
  command: bundle exec rake db:migrate && bundle exec rails s -p $PORT
  services:
    - yip-postgres

The name becomes part of the subdomain on deployment. The memory is kept low to keep the costs down for a personal project. The service listed should match the name of the database created in the space, above. Stick this in the repository so it can be used with the CI later.

Now the application should be ready to deploy with a simple cf push.

Continuous integration

I use Codeship, and their docs worked fine for me with two modifications:

• I dropped the CF_APPLICATION envar from the script as it's defined in the manifest file
• My first deploy failed as it couldn't find the required gems - subsequent deploys worked fine, despite a warning about including the .bundle dir in my repo (which I didn't)

Problems

This all works with minimal fuss, however I'm going to end up going back to Heroku. I originally discounted it because it didn't play well with Docker (a requirement I've since abandoned). Also:

• Heroku encrypts traffic for free on their own domain, whereas Cloud Foundry doesn't have this option. I can pay $20/month to use my own domain and cert but this breaks my first requirement. I can understand them charging for additional domain hosting but honestly, securing their own subdomains should be a given.
• The Cloud Foundry free tier database is tiny. Paying for a database adds a lot to the monthly costs - this is true of all the hosting options I looked at - so a useful free tier offering is important.
• Heroku is better supported. In Codeship, for example, there is a plugin to support it whereas Cloud Foundry requires a custom script. It's a simple one, to be fair, but it's symptomatic.
• The Heroku tooling and web interface are nicer. Again, unsurprising given how much longer Heroku has been around. The Cloud Foundry tools are fine, but the doing the same things with Heroku is just easier.

So there it is. These are just my experiences, based on not a lot of time and with the intention of hosting for a personal project.

Remember remember

Let's talk about fire.

I've always loved bonfires. They are primal and evocative - beautiful pillars of light and movement shining bright in darkness. So what does that mean for photography? Taking a picture of a bonfire is both an interesting technical challenge and an exercise in trying to capture the emotion of the moment.

Breaking down the requirements in photographic terms, what do we have? The fire is moving so you need a fast shutter speed to capture the tongues of flame. The scene is dark, but with a bright light source so you need to control white balance. A fast shutter means a higher ISO, to make the camera more responsive to the available light. A wider aperture allows more light in, but also restricts the telephoto options. So a first guess leads to something like:

• High shutter speed
• High ISO
• Wide aperture

So something like this:



Nice and warm, but the body of the fire is over-exposed.

What happens if we drop close the aperture and rely on a crazy-high ISO to capture the available light?



Loses the warmth, but now we can see into the body of the fire.

I wonder how important the ISO is in practice? What is we keep a wide aperture and fast shutter and drop the ISO?



There is more definition on the tongues of flame here when we zoom right in. Pushing the ISO distorts the image so this is a good change.

I started taking these pictures trying to logically get to some decent manual settings. Out of interest I brought out my phone and took a photo then looked at the results:



Huh. Seems nice. Settings?

• Medium shutter speed
• Tiny ISO
• Wide aperture

So the phone decided aperture was the key to sorting out the light capture, and didn't think the movement of the fire was too much of a big deal. Next step was to try this collection of settings on my big camera:



Different again. More warmth, losing definition on the tongues of flame.

And for a last hurrah, how about a different framing - adding some zoom (which narrows the aperture) and correcting a bit with ISO and shutter:



I have my favourites. It's very interesting seeing the same fire depicted so differently just by fiddling with some settings. I'm still calibrating my mind - looking at the subject and trying to imagine the different photos possible with different settings. The first few here I can say I thought through logically but I'm not sure I'd have got to the settings chosen by the automatic mode on my phone. I suppose I'll have to go to more bonfires and keep practising. Such a shame.

All the photographs taken that evening, including some different attempts at fireworks, are in this album:

Getting off auto

I take a decent number of photographs and as well as running a monthly photograph website I am taking it increasingly seriously as a hobby. Most of my photographs are taken using a camera phone thanks to the convenience of it always being in my pocket and, although I've dabbled with manual settings and know (very roughly) what they do I've never really been able to get consistently decent shots on manual mode. Never one to have a hobby and not get nerdy about the details, this month I got myself a proper camera and went with my Dad on a course to learn how to get off auto.

Buying a camera is hard. There are a load of nearly identical models and so much comes down to personal preference at a time when one doesn't know enough to have a preference. Given how expensive lenses can be, this is also a decision about which ecosystem to buy in to. It's a similar lock-in to the Apple / Windows or iOS / Android world in that it's reversible, but change can be tricky, unfamiliar and expensive. I may write about this in the future, but for the moment my key learning was the big choice if buying an SLR is Nikon / Canon. I ended up choosing a Nikon 5600.

Not all the way off auto

I've played around with manual photography before and have a (very) basic understanding of what the various settings do. When switching to manual mode on the phones I've used, the interface has encouraged taking full control - that is handling aperture, shutter speed, ISO and white balance all at once. This has always been too many interconnected, unfamiliar settings to learn and embrace at the same time. Possibly the single most useful tip from this course was to view it differently - take control of one of these aspects and let the camera do most of the rest automatically. This gives priority to a particular setting and therefore set of techniques (indeed, the modes on a proper camera are called aperture or shutter priority so ... yeah).

Aperture

We focused on aperture, letting the camera adjust shutter speed automatically and making corrections for exposure with the ISO setting. A wider aperture (aka a smaller f stop setting) lets in more light, allowing a faster shutter speed. It also allows control of the depth of field, enabling the blurred background effect used to create useful negative space and draw attention to a specific area of a picture.

This is often seen in portraits. Some useful tips for portraits - the eyes are the most important bit and should therefore be in the sharpest focus. So far, fairly obvious. Less obvious is that the camera may well actually focus on the closest part of the face - the tip of the nose - so for a really good picture some manual adjustment is necessary.

Another counter-intuitive tip is that portrait pictures are best taken on maximum zoom or with a dedicated telephoto lens. This is because the longer focal length allows for a wider aperture and hence maximises the depth of field effect.

Results?

And lo, the results.

Or, for a particular example of the portrait stuff above:


We had a lot of fun walking around Lacock and practising the specific techniques that fell out of playing with this one setting and the day did a wonderful job of demystifying the bewildering array of buttons on a decent camera. I'm looking forward to learning more.

Hanging up my abacus - part 2

It has been a while, but here is part two of my reflections on being a church treasurer. If you missed it, here is part one. Last time was mostly about vision and strategy. This is about people.

Back to the things I learned and experienced.

Lesson 5: It's dangerous to go alone...

Time is finite. An Earth-shattering discovery. However, the important point here is about setting expectations - both in others in yourself. This kind of role can easily be a full-time job, and this is one of the reasons it's often picked up by a retired person. Limiting factors like Having a Job and Wanting a Life will severely cut into the amount of time available and hence what can be delivered. It took me a long time to get this straight in my head and stop beating myself up for how long everything seemed to take - especially since business as usual work ate up a significant amount of the time I could otherwise have used for improving things.

It's impossible to create more time, so if you want to get more done you need other people to help. I set up a team to help and together we achieved things I had initially discarded as impossible for me alone. It astonished me that creating a Finance Team wasn't standard - I had several conversations with members of other churches who took this revolutionary thinking back with them. Another lesson for me was that just because something seemed obvious, that didn't mean it is. It is worth sharing your thinking.

Lesson 6: ... but choose your companions wisely

Forming a team in a charity is an art in itself. The obvious approach is to simply ask for volunteers, however if you do that you'll find you get the same faces who appear over and over in other teams. These people are going to be busy already, and are the people pre-disposed to volunteering - not necessarily the folk with the most useful skills.

Volunteers are nothing like paid staff. Our motivation is based on enthusiasm for the cause of the charity, supporting each other and our interest in the task at hand and it is all too easy to destroy any or all of these if the group isn't pulling in the same way. In my case, I was forming a group which was to act under my authority as a treasurer so I needed people I could trust to behave with integrity and who I could ask to do things and believe they would actually do them.

I avoided asking for volunteers. I think it misses the opportunity to encourage folk who wouldn't normally put themselves forward and it makes it very likely the resulting team will struggle to gel. Instead, after consulting as many people as I could looking for likely victims, I approached a handful of people who I thought would have the right combination of skills and perspectives. Most were people who did little to no volunteering in the church. I was fortunate that all but one agreed to join my merry band and they did so with the confidence that The Leader (me, gosh) believed in them.

There are definitely pitfalls to be avoided. Exclusively approaching people relies on your own knowledge of the people available, and triggers all of your biases, conscious or unconscious. There is a balance to be struck but it's too easy to just ask for volunteers and miss an opportunity to encourage people who otherwise wouldn't put themselves forward.

Lesson 7: A shared vision brings a team together

Next, I needed to turn the group into a team. Providing a constant stream of brightly-coloured cake definitely helped.

We had several sessions discussing the direction of finance in St Swithin's and prioritising out work. I outlined the problems as I saw them and let the conversation take its course and by the end we all owned the resulting plan. This bound the team together as the strategy came from the team, not from me. Not only did this do wonders for the motivation, we also ended up with a more ambitious programme than I had originally had in mind. When discussing the problems, we collectively decided that we couldn't do anything without a new computer system - something I had previously discarded as I didn't have time to do it. However, others in the group saw it was the most important thing we could do and volunteered to take it on.

Lesson 8: Learn to predict the future (and convince others of your mystic powers)

Accurate crystal balls are hard to come by, but there are ways to peer into the future with some accuracy. Looking at trends in income / expenditure give a clear indication as to what is going to happen - if the budget is any good it's quite easy to start thinking about what happens next year and the year after.

By asking ourselves several "what if" questions we got a picture of the problems hidden in the heart of our finance. Some were real (trends which predicted issues further down the line) and some were potential ("what if the top three donors all moved away this year?") and they let us set policies which helped mitigate these risks. For instance, following an analysis of the profile of our donations we diversified our income and changed the way we solicited giving.

In theory, spotting difficulties ahead of time should have let us do something about them before they became major problems. The hard part is convincing other people in the organisation when there are problems ahead. I presented year on year on the challenges hiding in the heart of our budget, however those problems were not immediate and so generating the right response (immediate action without panic) was extremely difficult. This is partly the nature of this role - as noted before, the ingrained culture was that finance was my problem so solving these problems was also my problem.

Regardless of the difficulties, all this helped me learn the value of investing the time in using the data available to make predictions, not just validate (or otherwise) the decisions of the past. Obvious, maybe, but this was the experience that cemented it in my mind. In the business of day to day life it's easy to forget it and drift further and further into being just reactive.

Lesson 9: Volunteers are not staff

I've already mentioned this above, but it's something to pick up on again. I was a volunteer in a professional organisation (the Church of England). There were things I had to do, legally, to perform my role. There was also an awful lot that I didn't have to do, but was very helpful to the wider organisation. As noted before, time is finite and whether I did any of these things depended on my motivation to spend that time assisting the wider organisation rather than (or in addition to) working to improve my local church.

Motivation is a tricky beast. With a volunteer, it's the only thing that is making them keep giving their time to you and not someone else. It is very easy for a large organisation to forget when it is relying on the goodwill of volunteers over whom they have very little actual control. However, if this does happen it doesn't take long for all those extra voluntary jobs to stop happening.

The short version is - if you're in a paid position, remember when you're dealing with volunteers or you will lose them. They owe you nothing.

Lesson 10: The importance of process

Volunteer groups are often held together by the links between people. Indeed, professional organisations usually have back-channels for getting things done because person X is friends with person Y in another team, which means they hear about stuff that is going on over there.

This is natural, and inevitable unless we want a world where we behave like automatons fitted to a job description. However, there needs to be a plan in place for the people who come next.

When I took on my role as a treasurer, it was on the back of all the previous finance people stepping down. I inherited very little, and while the former volunteers were very helpful it still meant I had to re-solve a lot of problems. Had there been clear processes in place I could have slipped into the role far more easily.

I'm cheating a little here as this was actually a lesson I learned back during my time as part of the team running a student radio station. Back then we worked very hard to ensure there was no clique (encouraging new people to join) and we had established processes (allowing others to keep the station going when we were gone). During my time as a treasurer, we brought in all kinds of new practices and we always did our best to make sure they didn't rely on the specific people in post. We documented, handed over knowledge as best we could and generally tried to make sure the next lot would have a far easier time.

I hope my successor has inherited a working system and is getting as much from the role as I did.

Sitting atop a mountain

In all honesty, August hasn't been a great month. It started well with a week at New Wine but has gone steadily downhill since then. That means I don't really have much to write about, so instead I'll tell a short story.

Once upon a time, some friends decided to meet. They had known each other a long time so one decided to bring the ultimate accessory - Mario Kart. Thus proceeded several races in which everyone realised they were much better at videogames back in the olden times.



But at least they looked like they were in a Nintendo publicity shot.

Alas, the restaurant decided they wanted to sell some food to the gamers and they ended up eating their own bodyweight in chicken wings, because otherwise the "all you can eat" deal just wouldn't have been worth it. But chicken was eaten and there was much rejoicing. That is, until someone pointed out that that restaurant was chosen because of something called a Freakshake.

And thus it was ordered. Several times.

What had we done?

Look at it.



LOOK AT IT.



And there were no survivors.

Next month I'm going to do something.

Homebrew postgres

> Hi me. Remember when you used to do all your development on a Linux machine? Wasn't that nice and easy?

> Well, no, but at least tools were consistent with the server.

> Ok, good point. But you're using postgres installed through homebrew now aren't you?

> Yes...?

> So how do you create a database now?

> Oh, well you since it runs as your user you can just do:
bundle exec rake db:create

With this in your config/database.yml:

default: &default
  adapter: postgresql
  encoding: unicode
  # For details on connection pooling, see Rails configuration guide
  # http://guides.rubyonrails.org/configuring.html#database-pooling
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

development:
  <<: *default
  database: mydb_development

# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
  <<: *default
  database: mydb_test

> That's remarkably easy.

> Mac!

> Traitor...

Hanging up my abacus - part 1

After three years as treasurer of St Swithin's church, Bath I have handed in my spreadsheets and passed the mantle to another. This also marks the end of six years as a member of the PCC - the council which oversees the church and acts as its trustees. It has been a very significant part of my life for more than half a decade, so I thought it worth looking back at the lessons I learned along the way.

I was made treasurer for two reasons. Firstly; people think I can do numbers. Secondly; there was nobody else. My first (and slightly flippant) question was "so how much money do we actually have?". As it turned out, attempting to rigorously answer that question in detail was the basis for the entire financial strategy for three years.

Lesson 1: Asking (then answering) the simple questions is a good thing

That first question was far too obvious to be useful, and yet proved to be absolutely key. Obviously it can be rephrased to something more sophisticated ("do we have a clear picture of the current financial position?") but honestly there's not much point. I spent a lot of time carving this question up in a variety of different ways in an attempt to understand the nuances and that was a great exercise to help map out the problem space.

The important lesson here was accepting that, despite this being a complicated role, asking simple questions was a valid way of approaching it - those simple questions had a lot of value.

And that leads to...

Lesson 2: Helping others understand the answers is a vital skill

As soon as I had to take a proper interest in the church's finance I had some basic questions. Of course, if I was asking those questions it was logical to assume that others would be asking them to me in my role as Guardian of All Money. And lo, they did.

Between frequent reports to the PCC and annual reports to the congregation at the general meeting I had to learn how to take a complex financial position and explain it in sufficient detail to a roomful of people who all wanted me to say "it's all fine" and then stop talking.

The key to this was in the narrative. If a talk is structured so it follows a logical, coherent thread then it is far easier to follow. There needs to be enough detail to engage people and back up the story, but not so much as to drown it. Given the nature of the subject (numbers - few people love numbers) then patterns need to be explained, and their significance highlighted.

I presented the financial position three times at annual general meetings and each year got extremely positive feedback along the lines of "at last I understand the finances!" so I think I did something right here.

There were a few lessons along the way. Accepting that other people actually did care about the same things I did; knowing the audience well enough to present the right information; and accepting that that audience really appreciated clear communication with enough detail to actually understand it.

Lesson 3: The power!!! AHAHAHAHAHAAHA!!!

The role treasurer is potentially very powerful. I had all the information about the finances and I drafted the budgets. Between those, I could heavily influence every financial decision made in the church - and most decisions have a financial component. That meant I could change church policy and promote my favourite projects by allocating money and it would be a challenge for anyone to question me as I was the one with all the knowledge.

To be clear - this is a situation I worked very hard to avoid. It was especially true for us - we had a year without a vicar during my term which gave even more scope for abuse of power.

The problem is, the roots are in the control of information - specifically the other members of the PCC and leadership not knowing the finances in enough detail and, despite what I just put in Lesson 2, most simply didn't want this information.

There is a point here about duty of care. A church is a charity and the PCC members are trustees, which means they have a legal responsibility for the finances - a duty they can't properly perform without clear information. It's not a duty that is always made clear and that is why there is a tendency to let the treasurer just get on with it, which is where we came in.

It is a different problem from the last one. That was about answering questions clearly. This is about encouraging the questions to be asked in the first place and that is a cultural shift - a fairly major one in our case. I ended up having to change the way we thought about initiatives and projects in concept - getting costings and financial thinking in from the start to help weigh up the relative merits (rather than deciding on a priority then working out whether we could afford something) and coupling that with changing the book-keeping processes to allow easily-produced monthly financial updates.

There is a lot I can write about the techniques I used here, but the important lesson is that there are some things people have to know, whether they want to or not. In that case it's important to make the knowing as simple as possible - ideally ingrained into process in such a way they pick up the information as they go with no extra effort on their part.

Lesson 4: Having a vision

These three points all link into transparency and communication - being very clear about what one is doing and ensuring people understand why. About three months into my time as a treasurer I wrote a vision for what I wanted to achieve and it could easily be summarised as "make it easy to get information" and "make sure it's not embarrassing if people do".

This document was primarily for myself, so I could get my thoughts in order before moving forward. It proved incredibly useful as something to which I could later refer in order to judge whether things were developing in the right way. It's not exactly insightful, but this was the first time I had written a long-term vision and the lesson for me was how useful it turned out to be. I also learned the difference between strategy and vision and how important both are, but also how to consider them separately.

None of this has touched on the other people involved, but this is getting long so I'm going to split it into two. More soon.