The year that was, 2016

Well, I think most people would agree that 2016 sucked. I'll certainly be glad to see the back of it. Aside from the various events we all know about, it's also the year I lost two members of my family. Focusing on the positives though, I have moved jobs and reconnected with a lot of friends many of whom I've not been in contact with for many years. The point of this annual post is to remind myself of things I do to flex my creative muscles and prove to myself that I don't just spent my time watching videos on YouTube and playing Gauntlet.

• 13 posts on this blog (14 including this one)
• another year of the Year in Pictures site, with doubled the number of contributors
• created a book out of the 2015 Year in Pictures posts
• contributed to a few open source projects, including libraries.io
• well over 200 contributions to my github account
• wrote and delivered two sermons
• wrote and GMed a roughly monthly 5th Ed D&D game
• restarted my instagram account
• loads of photos on my flickr stream
• updated my website and Gareth's website
• plenty of game recording, with sadly little to no video editing
• some small voiceover work

And I went back to doing some serious exercise and started playing a lot of board games again, not to mention the time spent acting as the Treasurer (and member of the PCC) of my church.

Resolution count: 2/10 - appalling.

I spent a lot of time on technical things and photography this year at the cost of writing. Next year I want to step up the writing and draw some more maps.

How to Mac

A while ago I made the switch from iOS to Android and now I find myself needing to make a similar transition - this time from Ubuntu to OSX. I've been using Ubuntu for development for around twelve years and like most developers I use a lot of keyboard shortcuts so to say this new world is scary and unfamiliar is more than an understatement. As before I'm writing this for myself in the future if I have to go through this again, and anyone else who has to do this so you know you're not alone.

Getting started

Out of the box nothing was too painful. Sure, the keys are in the wrong place but that is something I'll get used to eventually. I was shown how to put the mouse scroll the correct way up so that helped with moving around (it's in System Preferences -> Trackpad) and since to start with life was all about Chrome and simple text editing it didn't hurt too much. Except for the loss of function keys and the missing delete key (fn+backspace). Sigh. Still, the hardware is genuinely lovely and I'm very impressed with the battery life. As I type this I've been working all day from the battery and I'm still seeing 41% charge. Now; my Linux laptop has seen some serious miles but I don't remember it ever doing this well. Plus it's really nice having the operating system work 100% including hibernating, sleeping and all the other bits and pieces. That has been getting much better on Linux over the years but if I'm honest it's just nowhere near as good as I'm seeing here.

First shot at some real work

So then I had to start installing tools and getting things set up for actual web development. First up, Chrome. Gone are the days of hitting F12 to bring in the developer tools or a two button shortcut to view page source. Now I have to some weird contortion exercise for the tools and I have to include the command key to view source. Maybe it gets lonely and sad if it isn't pressed often enough? Anyway, my brain will remap this eventually so it's not the end of the world. 

Bring forth the command line

At the suggestion of literally everyone I immediately abandoned the default terminal in favour of installing iTerm2. That involved installing a package manager and a plugin for the package manager. This is weird territory for someone who is used to apt being an integral part of the operating system, but I was still impressed by the screen and battery life so I rode that happy wave a bit longer. First I needed some other odds and ends and eventually a friendly Mac user at work gave me some commands and I typed them in and Things Started Working (because when did just typing in commands blindly ever hurt?). I'm impressed with his wizardly powers, but I'm well aware that one day I'll need to do this again and I may be in serious trouble.

Command history suggests "we" did something like:

1. install xcode to get gcc
2. install homebrew: /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
3. install cask: brew tap caskroom/homebrew-cask
4. install iterm2: brew install iterm2

Well, if my memory has failed me it's a problem for future me. In the meantime I have a package manager and now an exciting terminal which is all the wrong colours, but that's hardly the Mac's fault. A few tweaks and I'll be up and ... wait, why is the window split in half? 

What do the buttons do?

The terminal is where the keyboard shortcuts really started to hurt. A combination of some of the shortcuts being different, an entirely new modifier key to contend with and my brain attempting to use CMD instead of CTRL except now apparently not all the time produced many, many frustrating mistypes and resulted in a couple of hours rebinding keys. It's now better - not as good as the default terminal in Ubuntu, but very usable and at least iTerm2 lets me save my profile for future use so I don't have to go through that again. 

I also discovered that OSX uses slightly different environment files to Ubuntu so I had to source my bashrc file from bash_profile to get that read properly. Apparently age-old Mac users keep build scripts so a new machine is just configured for them while they make tea. 

At least git works, right?

Yup - no problems here. Well, until I attempted tab completion at which point I was told that I actually needed to install something to make it work, which in turn needed me to reinstall git via homebrew.

Sigh.

1. brew install git
2. brew install bash-completion
3. git config --global push.default simple

That last is, of course, not a Mac specific requirement and shouldn't be needed since git 2.0 but I don't trust default behaviour since Code Vanished back in my last place of work and mild panic ensued. If you don't know what it does you should probably be doing it.

Installing SublimeText 3

I had already realised I wasn't going to be able to directly copy my old Sublime config to this machine as I had a whole new button to work around. Still, I assumed installation would be straightforward then I could spend some time remapping keys until I was more or less happy. I also wanted to be able to use the subl command on the command line to open files like in the old days, but this was proving difficult so I had to call in help (from a different helpful Mac user as I was already feeling like a moron at this stage). The look of horror he gave me when he saw how I was running Sublime told me I had done something wrong. Again.

It turns out that running a downloaded dmg file runs from a mounted volume. I needed to drag my running application to the Applications option in the file browser and then the Mac did some more exciting magic things and everything was fine. Right, not going to forget that one in a hurry.

Where are my windows?

One of the other things I really miss from Ubuntu-land is the way it handles multiple desktops. Maximising a window in OSX puts it into a full screen mode which moves it to its own desktop and makes it difficult to move around if you're used to ALT CMD+Tabbing around. There is a way around this - it turns out that if you hold ALT and click the maximise button you get the old "make big" behaviour and ALT+Green toggles back again.

If you want a keyboard shortcut (and you do) you need to install Spectacle (brew cask install spectacle) then you can maximise (not fullscreen) with CMD+ALT+f. To go back to normal size you can use CMD+ALT+z (thanks to yet another friendly Mac user for that one).

Fortunately, the CMD+` shortcut to move between windows of the same type still works, although harder to use with the § ` key moving to pastures new.

Is it working now?

I think so - or at least it nearly. RVM installation didn't bring up any nasty surprises and seems to be working perfectly. On the other hand, Finder is extremely odd. If, like me, you're used to moving around files with the arrow keys then hitting enter to open them, you might be surprised to discover that enter in fact lets you rename the selected file. If, like me, you then go hunting for the alternate shortcut you may struggle to find it unless (unlike me) you think to try CMD+Down which for some reason will do what you want. I had to ask. Now you don't have to.

These are the reports of my adventures so far. I may do a follow-up when I have to do battle with virtualisation if it proves tricksy (which is likely because, to be fair, it's hardly trivial on any platform) or if anything else thrilling comes up. After quite a few hours playing with this system I'm in a position where it is doing a fair impression of vanilla Linux. Albeit with the keys in the wrong places.

I hope this helps someone. Happy Christmas.

Edit: behold Part 2.

The first day

It occurs to me that, having been working at the University of Bath forever, I have experienced very few first days. For obvious reasons I've been thinking about working environments a lot recently, along with expectations from both employers and employees. The less-than-insightful thought being that the world would be much better if there was less fear. I wonder how many people would make positive changes in their lives, such as moving job, if it wasn't for fear. I know that I'm scared to be moving. Scared that the new place will reject me, scared that I wont be able to do the new job, scared that I wont like my new colleagues. None of these have any basis in reality. There hasn't been anything to suggest the new place will be anything but lovely and although the work will be different I'm definitely up for the challenge - plus they interviewed me and decided I am capable and they should know better than me at this stage.

So really my fear is based on the loss of my old job (which was full of lovely, talented people and a great environment) in the face of an unknown future. But moving on has been the right decision. It has allowed me to advance my career and re-evaluate my professional worth - both of which are Good Things for anyone to do. In turn, the university is going to have to face questions about how it employs developers - questions it can (understandably) avoid while it has people in post - also a Good Thing for the industry as a whole.

If movement is good, why isn't there more of it? That brings me back to fear and for the moment the first day. I know that one way or another I'll be uncomfortable on my first day and that is mostly due to my history of first days. I'm expecting the next one to be better and I'm looking forward to being involved in making them better for others when I'm the experienced one.

My first first day

My first job was as a lifeguard in a place which shall remain unspecified. Memories from that day involve arriving around 5.30am (eugh) and pretty much immediately being sent to set up some giant trampolines on my own. I later discovered that there are supposed to be six trained people involved in setting these things up. Fortunately I was rescued by some more experienced colleagues.

My second first day

My second job was at Unilever. It was a great job but day one was a mess. I was sent to the other side of the country, where nobody knew who I was or why I was there. I ended up interviewing people about a project I knew nothing about all the while wondering when I was going to wake up from the crazy dream.

My third first day

This was the first day working on the University of Bath Helpdesk, although the strongest memory was of the interview. I'd been sitting with a friend (who already worked there) fixing a laptop for him. The supervisor came over, saw what I had done and asked if I wanted to cover the next free shift. I was thrown straight into the action, with a small amount of shadowing an experienced colleague to show me how things worked.

I actually remember very little of this day so it must have been pretty smooth overall.

My fourth first day

My first day as a developer. I was shown to a small office which was about big enough for one and a half people. I was the half. Over the next few days I managed to cannibalise a working computer from various contacts around the university, including some flatscreen monitors from the dawn of time (the desk wasn't big enough for the more common CRT monitors). I managed to borrow a chair from a generous colleague in another office (he had two) then I was shown around the various systems on which I would be working - of which I understood exactly nothing.

Oh and the office let in the rain.

Not that I'm knocking this job. As I'll write about in another post I feel incredibly lucky to have had this opportunity!

No real conclusion here. I suppose the direction I'm heading is that if we want to improve our industry we want to encourage people to be the best they can be, which will likely mean enabling people to move around easily. One problem to overcome is the fear of moving and one of the things to fix there is the inevitably-scary first day. Each environment is different, but some basics (meeting people, first day activities, desk, computer, access) are going to be consistent and we really should have this nailed as an industry by now. So much of fear is the unknown - simply sending out a basic itinerary of the first day should help quell that.

The rambling story of how I became a developer - part 1

Just recently I've had the privilege of advising a few amateur developers on how to step into the world of professional development. I find this a difficult question but since working with and helping encourage those new to this world is very important and something I hope to be doing a lot more of in the near future I thought it best to get some thoughts in order.

How did I get there?

First things first - I can't claim to be any kind of career expert. My own tale has been a combination of providence and hard work, not particularly shrewd choices as I've progressed - at least not deliberately.

My first IT job was a summer spent as a business analyst, working through a huge data manipulation job and providing the technical expertise to the project manager. This wasn't why I was hired - I was supposed to be doing some kind of data entry as a holiday job - but by a series of coincidences I ended up talking to everyone who the project affected and accidentally doing some in-depth user analysis which led me to ask lots of questions about the best way to move forward. In my first job I learned the importance of the end users.

Next up, I spent a year in user support on a help desk, helping look after a campus full of computers. Again, lots of opportunities to talk to the end users and hear their difficulties and frustrations. This sort of experience is really important for someone who wants to be a good developer. Being able to create great code is important, but if you don't understand the people who will be using your product you will only ever be able to create to the specifications provided by others and that will limit your ability to be effective and put a ceiling on your career.

The help desk also gave me my first proper chance to effect change on my working environment. We had many processes which needed to be more efficient and I was fortunate that the people around me (and particularly my manager) were open to experiment and change. This is understood with the benefit of hindsight and experience - at the time I just had an idea, had a bit of a chat with my manager and gave it a go. Looking back I'm honestly surprised they gave me as much freedom as they did. Being able to critically analyse and successfully question the status quo is an important skill for anyone working in a team and especially in the rapidly-changing world of development.

The first summary

So far I think the key points (other than the rather obvious "make the most of your opportunities") are:

• get involved with the end users
• question the world around you

It's never too early in your career to ask "is this the right thing to do?" - it will probably be the most important question you learn. Of course, the other vital part of this skill is being able to ask without annoying and alienating your colleagues. While sometimes it is important to challenge authority or speak truth to power, or whatever the phrase is at the moment it is rarely a good idea to directly butt heads with people higher up the food chain. In a good working environment, questions and discussions should be encouraged (if you're finding you can't ever ask "why" then you're working in the wrong place) but you need to know how to approach such a conversation and when to back out.

Basically, soft skills matter.

More at some point.

Still at university

Breaking with tradition here, I’m going to ramble about work a bit - or at least some of the weird effects of working at a university. I’ve been thinking more about them recently, partly because I’m getting old and senile and partly because after 15 years I’m finally going to leave university and work Somewhere Else.

It started a few weeks ago when I left a meeting in the university library and walked across level 4. When I was a student (some 12 years ago or more) level 4 of the library was our territory. It was where the maths textbooks laired, and called us to gather even if we never actually looked at the things other than to marvel at their number and the amount of dust that had accumulated since the last pilgrims had arrived. At that time the floor was filled with bulky computers, with CRT monitors perched atop them like the rock piles in Blair Witch, and laid out in uniform lines which both made economical use of the space and sapped any will to live from you. Now though? Well, now it’s…



Well, exactly the same really but with more modern tech. The weirdest sense of deja vu. I have to wonder if the Silver Fox is still haunting the place, seeking those students who dare to consume something other than blessed water in his hallowed halls.

This time-bubble warps perception everywhere and our relatively low staff turnover only encourages it. This week is Freshers’ Week, which means hundreds of school leavers are roaming the grounds in an attempt to find the fabled “north buildings”. They are aided in their quest by the returning students and it is so very easy to look at these second and third years and relate to them, thinking “that was me not so long ago”. And yet it was more than a decade since I was a third year student, worrying about coursework and helping run a radio station. I have many friends who have had children - several children - in that time who are now going to school and looking at us as the uncool adults we really are.

It is a frog on slow-boil problem I feel (minus the brain-removed issue those frogs experienced, although…). I haven’t left so I haven’t aged. Despite doing adult things like getting a mortgage, life has failed to convince me that I have become, in theory at least, a responsible member of society and A Proper Grown Up. Maybe it is because I haven’t had that moment, a variant of which I assume everyone else goes through, where I suddenly become my Dad, understand what’s going on in the adult world, learn to appreciate sport and discover the enjoyment in gardening.

Yesterday, I was sitting in the Plug (the student bar) drinking diet coke and MMMBop started playing over the speakers. Aside from the video being projected onto the wall (and the lack of a half-completed piece of maths tutorial work), this could have been a scene from 15 years ago - except I was a bit fitter (although I’m more flexible now - in your face younger me) and had a bit more hair. Fortunately the modern world dragged me back from my time travel experience as the bar audio has a feature which lets anyone add music to the playlist from their phone anywhere on campus.

Back in the day nobody played anything but URB if we had anything to say about it.

Jekyll and the build scripts a few years on

A few years ago I moved my sites from a PHP templating system to static generation using Jekyll. How is it working out?

Pretty well. I’ve had no downtime (that I haven’t caused) which is to be expected on a low-traffic website serving HTML files. Updating content and templates has been easy, with Jekyll remaining simple to use. While I’m sure I’m in need of an update, that is less of a concern than if I was running code exposed to a user. Overall no problems with the technology or maintenance - indeed I find it much easier to work with than previous versions of the site as I don’t have to re-learn my configuration each time I want to do anything more complicated than change some words.

The biggest win - and something I actually considered skipping when I initially implemented - has been with the build scripts. In professional life I wouldn’t think twice about writing automated build scripts for a project but we all know that this kind of thinking isn’t as rigorously followed for personal projects. I wrote a simple mina script for deploying (and updating) my sites and several years on I deeply thankful to my past self. I haven’t had to keep my build process in my brain at all - just the magic command, which is in a README somewhere. This has meant small updates have been easy, the most boring part of site maintenance has all but gone away and consequently those updates have actually happened.

The lesson to take away here is that doing the hard (and dull) work up front of defining a development process and writing deployment scripts was worth it. Not so much because of time saving, or the consistency inherent in an automated process - but because these benefits actually encouraged me to maintain my sites in a way I simply wouldn’t have done had I been required to remember how to deploy my work each time I did anything.

Printing a photo book

TL;DR - I used CEWE Photoworld and it was good

I have been running a photography website since the start of 2015 and I thought it would be nice to turn the pictures from last year into some kind of coffee table photo book. That's 72 photographs (12 months x 6 photographers) each with title and caption, plus each month needed a cover page, and I wanted the first photo of each month to appear on the right-hand page so I needed another page per month to shuffle the photos on. That's 8 pages per month, for a total of 96 pages to lay out - not a small amount of work so I needed a site which would give me a tool which I could use without wanting to do Very Bad Things by the end of it.

Photoworld

After some searching I ended up using CEWE Photoworld for three important reasons:

• They have a desktop tool
• They have an online help which actually ... helps
• They have a 100% satisfaction guarantee

The desktop tool is hardly the pinnacle of software design, but it does the job. It is quick, reassuringly responsive and allows text with background colour and limited bulk formatting. It let me put everything together in the way I wanted, including guidelines on where I could push content to the edge of pages and where it wouldn't work. This put it a huge step forwards from Photobox, which I tried first because it is the famous one. Photobox offers a web based application which didn't let me add text with a coloured background and generally had that cumbersome feeling of web applications from last decade.

The Photoworld online help includes a real time chat, which was very helpful. I used it twice. The first agent was very responsive and helpful, answering my questions and generally being very reassuring. The second one was significantly less so - I got the impression he had far too many simultaneous conversations running, and was annoyingly vague when I was asking very precise questions (if you've got five different types of paper to choose from then you really can't use terminology which ambiguously covers three of them when recommending a choice). Still, we got there and the site helpfully emailed me a copy of the conversation which I kept just in case I needed to trigger the guarantee at a later date. Fortunately I've not had to test their guarantee in practice, but it was very reassuring to know that I had that safety net.

Reassuring is, I think, the key word for describing dealing with Photoworld. They know their service is expensive (as in book printing is expensive - I don't think they are expensive compared to their competitors) and is likely to be bought by people who haven't got a clue what they are doing so they do their utmost to make you feel like you're in good hands, and minimise the chances you're going to make a mess of what you're doing. One example from the site text - each book passes through 15+ pairs of hands as it is produced so it is thoroughly checked for imperfections. Regardless of how helpful this actually is in reality, it is an encouraging thought.

The only time I feared for my book while using it was when it came to finish and pay. At this point it uploads the pictures and send you off to secure payment, or crashes horribly if you attempt to use the Paypal option. This is slightly frightening when you've spent tens of hours laying everything out and proofing the book and all of a sudden it looks like it might be stuck on your desktop for all of eternity. Anyway, a switch to using a credit card bypassed that part of the application and it all worked fine.

Behold

The book arrived slightly quicker than promised and looks great.



The presentational box was an extra, but looks really nice.



And the photos printed well. There is a notable variance in the quality of the pictures between the different photographers, but that is to be expected, reflecting the different cameras in use.

Overall, I'm impressed. I'll be using Photoworld again.

And a big thank you to Kirsty Davey for proof reading it and correcting my mistakes. If she had a web presence I'd link to it.

The Brexit post

So, like everyone else with a social media account I have an opinion on Brexit and the chaotic aftermath in which we find ourselves with both the government and the opposition collapsing in on themselves at exactly the time when some actual leadership is required. While I doubt I've much new to add, one day I will look back at this blog and I want to see a collection of my thoughts from this time.

Disclosure first. I believe in the Europe Union and the European vision. I believe that as a nation we are more than this small island and that not only means we should engage with European politics, but we have a responsibility to do so. So yes, I voted Remain.

Obviously I think the referendum result was a terrible decision and I'm appalled at the lack of conviction shown by the winners in the aftermath - be it Boris deciding that after leading Leave, he doesn't want to lead actually leaving or the calls from the Leave camp to put off invoking Article 50 for an unspecified amount of time. The indecision and lack of any coherent plan for this result is, frankly, terrifying.

Remain supporters are trying to process the situation. Some are calling for a second referendum, while others are looking to Scotland to find a magic veto and dig us out of this mess. Still others are looking to claim citizenship of other countries, or leave altogether. There is a hope the government will simply ignore the result, which seems a reasonable reaction, if wishful. It's not like they've listened when it comes to anything else recently.

Many have had enough of all this. They've sat through months of campaigning, of impenetrable rhetoric, half-truths, scaremongering and downright lies and, understandably, just want to get back to normal life. They want cats and babies on their Facebook feeds, not endless discussion of what is seen as a now-closed issue. This resignation hasn't gone down well and others are asserting their right to be angry, leading to a weird meta-argument.

Personally, I'm sympathetic to the weariness. I'm tired of all the debates and all the fighting being about stopping things getting worse. The Remain campaign wasn't about fighting for a better future - it was a rearguard action to defend what was the current (far from ideal) state of affairs from the self-serving and deluded. The same as the battle to stop the NHS being taken to pieces and privatised. And the battle for the BBC. And the schools. And the Snooper's Charter. And so on.

The left does not seem to be fighting for improvements any more. We aren't campaigning for positive change, but opposing negative change which rather plays to the whining liberal stereotype and it is really hard to get gain any kind of momentum when your message is "now, hang on". It is at this point we really need something big and positive we can get behind in the political arena. We should be able to look to the opposition for some kind of balance. Except the opposition has struggled to be credible for the last few years and has just imploded.

This is, of course, an emotional reaction to the current situation. There is a tremendous amount of work done by those who are campaigning for a genuinely better future, and I am doing a disservice to those fighting the rearguard action. But ultimately, major change will need to come through voting in what I am going to crudely call "better people" and that means increasing engagement in a process which for me (someone who is already engaged and interested) is currently a source of helplessness and fatigue. I doubt I am alone in feeling this.

I hope future-me reading back can say that I've played a part in improving this situation.

Exporting a postgres database from Heroku and importing to local install

Continuing with my efforts to learn some basic, useful postgres admin commands it’s time to look at importing and exporting data. We are going to export a postgres database from Heroku and import it to a local postgres install for development.

I’m assuming the Heroku toolbelt and postgres are installed locally and myuser is already created. I’ve written some very basic pointers to (local dev) postgres installation and administration already.

We are going to export the database used in myapp and import it locally to mydatabase to be owned by myuser. Brace thyself.

Export from Heroku

This is the easy bit.

heroku pg:backups capture --app myapp
curl -o latest.dump `heroku pg:backups public-url --app myapp`

Boom.

Import to local

We are going to use the pg_restore command, but that needs to import as a postgres superuser. It will also prompt for a password, even if the user is set up for peer authentication (as per my last post) so we’re going to create an importer user with superuser powers. There is probably a better way to do this, but life is short…

Logged in to postgres as a superuser:

CREATE USER importer WITH PASSWORD 'mypassword';
ALTER USER importer WITH SUPERUSER;

We also need a target database:

CREATE DATABASE mydatabase;

Then to import the database (back on the command line):

pg_restore --verbose --clean --no-acl --no-owner -h localhost -U importer -d mydatabase latest.dump

This will throw some errors when the DROP commands in the Heroku export fail. This seems to be ok, but check nothing else has gone wrong. There is probably a way to have Heroku export the database without the drop statements to eliminate these messages.

Back in postgres as a superuser, switch to the new database and assign the correct ownership:

\c mydatabase
REASSIGN OWNED BY importer TO myuser;

Done!

HTTPS for a small site

We all know it’s a good thing. Security, SEO and soon not being called out by Chrome and Firefox for being insecure. But for a small, personal site it’s a pain in the rear to set up and the certificate is prohibitively expensive, right? Right?

Maybe not. Let's try and change this:

The certificate

These days you can get a 90 day certificate for free from Let’s Encrypt, which is news to me and the reason I thought I’d give this a go.

Main stumbling block removed.

Apache config for SSL

Ok, I can write this config myself. However Let’s Encrypt has a magic tool which claims to do everything for me. Let’s find out.

git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt
./letsencrypt-auto --help

• It downloaded a python environment for me.
• It did a thing with root privileges courtesy of sudo. Probably shouldn't have used a window in which I’d previously sudo'd something. Oops.

./letsencrypt-auto --apache

The automated thing doesn't detect my domain. It detects a load of others, but I’m not ready to destroy those yet. Boo.

Also, I'm guessing with letsencrypt-auto. It seems to pass flags to the letsencrypt script which is buried somewhere. Turns out I am right. Great.

I have to agree with the T&Cs to register with the ACME server. Aside from the obvious, ACME seem to mean Advisory Committee on Mathematics Education which I don’t think is relevant here so clearly I am getting a cert from the same people who supply anvils to Wile E. Coyote.

Seems legit. Let’s do this.

./letsencrypt-auto --apache -d tomnatt.com

Still not finding my domain. Is it … confused by the number of domains? Nope, it doesn’t like files containing multiple vhosts. Oh. Reconfiguration time.

...

Ok, updated. Now time to fire this baby up. The original command now finds all the domains. Go! What could go wrong?

Wrrrrrr…


Well, shit.

Minor problem - apparently I'm loading my fonts over an insecure connection.


Bingo.

For those of you not up to speed with the arcane art of reading browser URL bars, the shield is gone which means the browser isn't blocking assets trying to load into a secure page over an insecure connection.

These certs expire in 90 days so time for a simple cron.

00 03 * * * $location/letsencrypt/letsencrypt-auto renew >> $location/letsencrypt/logs/renew.log 2>&1

Docs recommend checking daily, so that should keep things up to date. And potentially fill the filesystem. Meh.

So, my site is available over a secure connection. Hurrah! The “ensure all connections” setting seems to have set up a basic redirect, which is good although I'm going to have add the HSTS headers myself and hope that doesn't get toasted when I next run one of these scripts. Renew seems to behave though.

HSTS

HSTS removes a vulnerable step when redirecting from an insecure to a secure connection. Details on the magic can be seen on the OWASP site.

The important bit of Apache magic is:

Header always set Strict-Transport-Security "max-age=31536000; includeSubdomains; preload"

Which is stuck into the https vhosts and requires mod_headers enabled.

Testing this was a world of fun. I'd recommend disabling the cache (in the dev tools), using a plugin to inspect the headers (I like Live HTTP Headers) and making liberal use of this secret page to check the status of the HSTS settings. This is all in Chrome.

Tidying up

It seems only the automagic script doesn't like my old Apache config. Now it’s all set up I can put everything back in the same file.
So now I am handling four different connections in the same file:

1. https://www.tomnatt.com
2. https://tomnatt.com
3. http://www.tomnatt.com
4. http://tomnatt.com

With 2. and 3. redirecting to 1. and 4. redirecting to 2. so as to pick up the extra HSTS headers.

HKPK

Yeah, that can wait.

Overall though, this was not the trial I expected. Getting a cert is now really easy. The only parts that required any real thought were figuring out how to arrange my Apache config and checking the HSTS headers were being set correctly.

No excuses any more! Best do the others.

Creating a database and user for local postgres development

So, yeah. I'm a postgres n00b. But I'm a n00b who wants to be able to create a non-superuser account and database, relate the two and also be able to remember how to do this again in two weeks time.

Connecting

As superuser via UNIX user authentication:
sudo -u postgres psql postgres

As superuser directly (-W forces password prompt):
psql -U postgres -W

As a user to a specific database:
psql database -U username -W

Basic commands

Show tables:
\d

List users:
\du

List databases:
\l

Quit:
\q

User management

CREATE USER username WITH PASSWORD 'password';
DROP USER username;

Database management

CREATE DATABASE database;
DROP DATABASE database;


Granting ownership and permissions

ALTER DATABASE database OWNER TO username;
GRANT ALL PRIVILEGES ON DATABASE database TO username;
Dumb settings for local dev.

Also, if you're getting problems connecting try replacing local peer with local md5 in /etc/postgresql/VERSION/main/pg_hba.conf.

Much of this came from this post. I'm planning on using pgAdmin3 as a database explorer when I want something quicker than the command line (on Ubuntu).

And lo, I have Windows 10

I’ve been meaning to upgrade my gaming PC to Windows 10 for some time but it didn’t manage to be the most important thing on my todo list at any point. Partly this was fear of the unknown - I knew Win10 was going to be a shift in UX and also thought it likely to break at least one peripheral. My attitude to an operating system is that it should do its job quietly and not get in the way and, frankly, I didn’t feel inclined to invest time in adoption pains. That’s time I can spend more profitably sleeping or looking out of the window.

Microsoft, it seems, had other ideas. They pushed the Win10 upgrade through their patch management system and I fell victim to the auto-upgrade problem. It’s a dark, stormy night. The wind is shaking the windows, drowning out the drumming of the rain. I’m sitting in a partly lit room, curled up comfortably and reading something on my tablet. In the corner, my computer is on, untouched for the past hour. I glance up and a chill runs through me. On my monitor is the ominous message “75% upgraded”.

I could write extensively about the aggressive way Microsoft have pushed Win10. I could complain at length about it arriving on my computer unwanted and the abuse of trust around using a security patch mechanism to automatically install a complete operating system without my input. I could compare the techniques used in release of this system to the way malware is spread. But others have done all that. Instead, I’ll focus on my experiences now it has arrived.

It’s fine.

Sorry, that was really dull but honestly it sums it up. The installation process was really simple. I had to track down and turn off the P2P patch sharing stuff (uncharitable, but I wasn’t in the best mood at this point) and some of the information sharing stuff (Win10 is horribly intrusive) but otherwise it just loaded up as New Windows with no real fuss.

The next evening I sat down to see what had really happened behind the scenes. First step was going through the security and privacy options. The defaults here were horrible (everything seems to have access to everything, including cameras and microphones) but the menus themselves were clear and it was easy to turn it all off. I also came across some advertising options - it seems in the brave new world of Windows it’s a good idea to have (targeted) advertising on your lock screen. Fortunately, both the targeting and the advertising can be disabled (separately) and so that went too. The start menu was a mess, but simple enough to remove the new and exciting rubbish and simplify back to the applications I’m actually going to use.

Next up, there is Cortana. I like the idea of Cortana and I quite fancied playing around with her. Unfortunately, in order to be helpful she looks at everything you do and sends it all off to Microsoft HQ so they can tune her electronic brain. So she had to die. Killing her off was actually harder than it needed to be - stopping her talking to Microsoft wasn’t too hard, but that left her zombified husk on my task bar and I had to work out how to purge her from there too.

Having finished with my electronic holy water, I moved on to my own customisations. I found that Steam, Chrome and Office all worked fine which is the majority of my use of that computer immediately. Also, my automatic backups (I use Macrium) continued to work and mapped drives were still mapped.

So far, so painless. I hadn’t needed to reconfigure anything and the new interface hadn’t caused me any real suffering. Time to check the two things I feared would break - the main reasons for putting off the upgrade in the first place. My joystick and my game recording setup.

First off, the joystick. My basic fear was that the (already shoddy) performance of the drivers would be even worse under a more modern operating system. My fears were confirmed when it failed to load properly. To Google! Fortunately, I wasn’t the only person looking for help (this thread was very useful) and - much to my surprise - Mad Catz had released some beta drivers for Windows 10. The Win7 drivers were released in 2011, whereas the Win10 drivers came from August 2015. And they worked. Probably better than the older drivers (I didn’t, for example, suffer any blue screens while installing them). I’d lost some of my settings, but that was easy to replicate and it was fine.

I did notice a problem on boot. Win10 boots faster than the USB devices which caused problems with my stick. This was easily fixed by disabling Fast Boot. It didn’t seem like the best solution, but it worked.

Next up, game recording. Astonishingly, this also Just Worked. Mostly. I had to re-enable some of the output devices in the sound menus, but I got everything going just by double-checking the everything in my original post.

Windows 10 is fast, stable, not overly ugly, and very easy to install. It’s a change to the user interface, but not one that particularly gets in the way of just using the computer. It’s a pig for privacy, but you can turn all that nonsense off. So, overall a surprisingly good experience. 9/10. Would have my computer hijacked and a new OS forcibly installed again.

Into space with the Saitek X52 Pro

Since Christmas I have been playing a lot of Elite Dangerous. It’s a great way to spend time - floating around in space, deciding what to do with an evening, heading off to achieve things and gradually increasing in rank and skill.

I cut my teeth (whatever that means) playing on a keyboard and mouse setup, which is … functional. At best. Online People say that a HOTAS setup changes the way the game plays entirely and is a must for any serious Elite player so I thought I’d give that a go.

After much deliberation (should I spend £270 on a replica of the flight controls from an A10?) I decided to go for the Saitek X52 Pro. It was, apparently, the stick used by Frontier Developments when designing Elite so should have good in-game support. There is a strong body of opinion that it is better than the newer stick, the X55, in terms of button placement and general feel (and saves £50 too). Plus it looks exactly like the joystick your avatar is using in the cockpit of your ship.

The good

• the hardware is lovely - solidly built and satisfyingly weighty
• ergonomic stick, adjustable and comfortable
• button placement is equally good with most functions falling naturally under my fingers
• I keep finding buttons - after a month of using it I suddenly discovered a small wheel on the throttle I hadn’t noticed before

The bad

• the drivers are horrible - I mean really horrible
  • it took several attempts and a few blue screens to install
  • I have to plug the joystick in to the SAME USB port - I’m not quite sure how they’ve achieved that
• the control software is horrible, although less than the drivers
  • saving the profile doesn’t seem to work properly
  • I have to manually tell it to load a particular profile before playing
  • in Elite some buttons can only be mapped after changing the default bindings in the profile
  • for some reason I seem to need the control software actually open to make some of the remapping work in-game

This is running the latest official Saitek / Mad Catz drivers on a Windows 7 machine.

So, did it change my life?

Well, kinda. It really has made a difference in game. I can perform manoeuvres that were next to impossible with the keyboard / mouse combo. More importantly, the feel of the game is indeed very different. The joystick and throttle really help with the immersion and even routine activities are a lot more fun.

On the other hand, the driver problems really tarnish the experience. I would struggle to recommend a Saitek device to others - especially since I’ve apparently got away lightly (the control software rarely crashes for me and my system remains stable). None of these problems are insurmountable but, basically, I expect a lot more from a piece of hardware costing in excess of £100.

I’m happy with where I am now, but it was far more work than I wanted to go through for a premium peripheral. If I decide to buy a new stick in the future I will be reading about the software support very very carefully before selecting my product and it will take a lot to convince me to buy anything with software by Mad Catz again. It’s a shame because the hardware is really very nice.

MyFitnessPal and gaming your weight

This year one of my resolutions was to take better care of myself by eating better and doing more exercise. I decided to use MyFitnessPal after a long and scientifically rigorous investigation (some people at work were using it and said it was good) and I’ve been fascinated to see how it has used some simple (mostly gamification) psychological tricks to focus and encourage me.

Your FitnessPal is watching you

The basic principle is simple. You keep a list of the food you eat and the exercise you do and it gives you a running total of your calories compared to your daily maximum (defined by level of activity, age, etc). At the end of the day, if you’re under your limit, you’re given a cheery prediction - “if every day is like today in five weeks you’ll weigh blah!” It’s surprising how addictive that affirmation can become. I want to achieve that, and apparently I can. I just have to keep going.

And there is an opposite reaction in me - I don’t want to disappoint the thing. I’m not sure exactly what happens if you go over but I suspect it gets mad and melts your phone. I haven’t dared find out since it got angry with me for not eating enough to live (this was a mistake not a conscious choice! Don’t do it kids!).

Food as a game

Not only does the app give you a report at the end of the day, it presents a running total so you can watch your calories slowly creep towards your daily goal. With the electronic Eye of Sauron always watching (and remember this thing is on your phone so is likely in your pocket all the time) there is a strong motivator to make you seriously consider that piece of cake.

However, if you DO eat that piece of cake then help is at hand. I know a lot of people who diet and are in a constant state of guilt over what they are eating. Clearly this isn’t good for their mental health; a calorie counter gives a definite indication of whether that snack mattered or not. It allows you to build in space for cake, or recover from eating it in a practical way without pangs of guilt. Drink water instead of wine with your evening meal and it evens out, no problem.

The application reduces the food you are eating to a series of numbers. As a mathematician and a gamer I like numbers - they imply a system and systems can be manipulated. Calorie intake can be substituted as above or changed via portion control. Everyone knows about portion control, but watching the numbers change on a screen makes it live for me and encourages me to actually do it. Doing some exercise raises your max calorie intake for the day so I find myself going swimming to give myself an increased ceiling. I like swimming so it’s hardly onerous, but I doubt I’d bother going as often as I do if I didn’t have this little numbers game running in my pocket.

Now obviously there are also unhelpful ways to game this system. You can, for example, eat at Subway but list your foods as homemade equivalents, thus saving yourself at least a hundred calories a sandwich. Or you can go for an extreme (ie plain stupid) diet - my calorie intake is fine if I eat nothing but 12 Cadbury’s Creme Eggs a day. However, in the manner of an 80s kids cartoon, there is a moral - if you cheat your Pal, you’re only cheating yourself.

The proof of the pudding

There are strong motivators to Do The Right Thing however all this falls apart rather quickly if there aren’t any results. So far I can report that it seems to be working. I’m obeying my electronic taskmaster and seeing the results, which encourages me to continue to do so - and so it continues until the machines are completely in control…

The app itself

A quick note on MyFitnessPal. I’m using the web based version and the Android app. It’s hardly the only calorie counter out there but it’s working well for me. I like the cost (it’s free) and now I’ve been using it for a couple of weeks I find it very easy to use. It did suffer from a pretty horrible start though. Before the app seeds its internal listings using your regularly-selected foods adding anything is incredibly laborious - which means the worst few days of the experience are the first few days.

On top of this, the home screen is initially utterly useless - full of news I don’t care about and pseudo-tweets which gave me a heart attack until I reassured myself they weren’t being added to my Twitter account. Eventually I discovered the option to turn this nonsense off and now the home screen gives a load of nutritional information which is actually quite interesting.

So I’m pleased with my experiment with a calorie counter, and MyFitnessPal in particular. I had serious doubts on starting - I thought the book keeping would drive me insane - but actually so far I’m finding it nothing but a benefit.