Monday, 6 May 2019

Buying code

TL;DR - I found this old post about outsourcing and found reading it again interesting.

The magic bullet that is outsourcing rears its head every so often. Why worry about organisational capability and capacity when you can commoditize the process? Push some money at the problem and it goes away, right?

Well, no. Not even close. There are many problems fundamental with this way of thinking and for me they boil down to two key things:

1. Code is never finished
2. Effective and sustainable outsourcing is really hard

I'm going to write a separate post about the first point. Probably more than one - every time I sit down to write something it grows by a frightening amount. The short version is that written code is the start of the journey, not the end. It's why we have entire frameworks (such as ITIL) to manage the services created. You may have paid someone to create the software - but then what?

I was going to write a series of thoughts to cover the second point. But then I discovered something fun - I've been writing on the internet long enough to be repeating myself. So here is me, nearly six years ago, writing about the pitfalls of outsourcing on my old work blog. It's worth a click - if only so they see a weird spike in traffic on an older post.

I find the thoughts there interesting for a few reasons. Firstly, the world hasn't really changed. At all. The fact that I'm writing this post is due to this approach still being seen as an easy solution, and still having some very dangerous long-term effects. The only point in there I think has improved is that Agile working with a supplier is a bit easier these days - although that might be a reflection of the difference in agencies operating in London and Bath.

Secondly, I find it interesting that I felt strongly enough back then to write a post about it. I was a developer, writing code and helping shape a university department. I saw problems with outsourcing, raised them, and when things happened anyway (and, despite the glib phrasing, I should make it clear that my concerns were addressed in the context of the department's priorities) I was one of the people who was saddled with the fixing and learning to make everything work.

These days I'm the Head of Software Engineering for GDS (for the moment at least) and I get to inflict that fun on other people. Or, more accurately, I get significant input into the operating model for the department - which means I need to make sure we're taking into account the points past-me made and ensure we're effectively taking them into account when considering how we build things. Because otherwise, I'm inflicting that fun on other people.

Saturday, 27 April 2019

It's dangerous to go alone

I've played Dungeons & Dragons, alongside other roleplaying games, for most of my life. In the last few years I've been telling people that the skills learned from playing (and especially running) this game are incredibly useful in the office. And people think I'm mad.

Time to explain.

Firstly, for those who don't know, a roleplaying game is a co-operative game focussed around collaborative storytelling. There are hundreds of different rulesets and different ways of playing those games but essentially the Games Master (GM) tells a story. Everyone else takes the role of a single character (the player character, PC) and describes what that character is doing, and how they are responding the challenges of the world. Imagine telling someone the story Lord of the Rings, but instead of passively listening the audience has control of the behaviour of Aragon, Gimli and Legolas (or, if you don't want to imagine it you can read this happening at DM of the Rings). The GM provides context in the setting and the reactions of other characters. The rules define what the characters can do and gives a framework for testing for success.

There are some obvious skills which are gained by playing. Most of the systems involve some basic maths that needs to be done quickly on the fly, which helps with arithmetic. There are social skills from playing in a group, and needing to work together. There is the creativity of thinking up the details of your character, including their motivations and history and some basic acting involved in their portrayal. In fact, Dungeons & Dragons (D&D) was used as a learning tool in schools before it was deemed Evil for reasons which are too tiresome to explain here.

However, I think there is a lot more to it than this.

Consider a job interview. A very common question is along the lines of "imagine you're working here - how would you respond to X happening?". I've had many conversations with people who find it very difficult to put themselves in a fictional situation, understand the limits of their choices in that scenario, and decide on their actions within that context. It's a pretty unnatural thing to do and something of a niche skill, but is definitely a barrier to getting many jobs. It's also exactly what you are doing almost all of the time in a roleplaying game. Pulling out your dice and declaring you're going to stab the manager probably wont help you in interview, but having the confidence to think clearly and quickly in that environment definitely will.

At the heart of many roleplaying games are the challenges. How do we cross this ravine? Can we get the gem from the sleeping dragon? The realm is being invaded, what can we do? These all boil down to "how do I solve problem X with resources Y?" My career has been in technical delivery and management and this has been the question posed by work every single day for over a decade. Like in the games, the answers have varied enormously, and often involve multiple steps to grow Y or redefine X. Practising these thought processes in a safe environment has been very beneficial to me.

Solving problems in D&D often involves communicating with other people (played by the GM) and that involves thinking about their needs and desires. And now we're practising empathy and understanding - in higher stake scenarios and closer to actual conflict than many of us will ever encounter in real life.

When one is the GM and running the game, all the above is dialled to 11. Instead of a single character to track, there are potentially dozens and you have to remember exactly who knows what and how those people might communicate. This starts to sound like the communication that happens in an office which is very important at times of change, or any time information needs to be passed around.

A large part of running the game involves describing the scenario and keeping the story moving. This involves a myriad of narrative and communication skills, learning when to focus on details and when to skip over for the sake of pace. What's important? What makes things come alive? Being able to tell a story is important at interview, and important for talking about your latest great idea in the office and convincing others of its greatness.

Taking a step back a little, being a good GM involves keeping the game itself moving forward. That means taking a group of people, listening to their thoughts and discussions, providing information where required but also helping them come to a set of decisions. All the while, you need one eye on the clock (unless you really like playing to the small hours and getting nothing done) and to keep monitoring the room to make sure that everyone is included. These are key skills required to run a successful meeting.

I can go on, and probably will at some point in the future. For instance, I haven't mentioned anything about teamwork (D&D is usually impossible if the players aren't working together). I look forward to the day when management training is a group of candidates running through the Tomb of Annihilation.

I think playing roleplaying games has the potential to teach some very positive skills. But if nothing else, it is a good excuse to own a lot of pretty dice.


Friday, 29 March 2019

HTTPS for a small site - redux

A few years ago I set up LetsEncrypt for my sites, so I could create certificates, do HTTPS and blah blah security. Anyway, it all worked well until, nearly three years later...


Still, an automated process that has been running unattended for three years suddenly stopping working has never caused a problem, right?

It turns out some things have changed with LetsEncrypt in the last three years. Like, everything. Everything. Even the name. Now the thing I want is called Certbot. Fortunately (and against the trend in modern days) it has improved with time. Now there are packages and guides! Sadly, migration is going to be worse than setting things up in the first place. It's installing new software then making sure it's renewing certs generated in the old way, in the old place.

Sigh, here we go.

So I followed the install section of this guide then ran:
sudo certbot renew
And it ... just worked?! I don't understand. This is not computing.

Ok, I need to fix the cron too. That'll cause problems.

Starting as:
export PATH=<boring path stuff> && /another/path/letsencrypt/letsencrypt-auto renew >> /path/to/logs/renew.log
Change to:
/usr/bin/certbot renew >> /path/to/logs/renew.log
And ... that appears to just work too?!

Amazing. It looks like some things do get better.

Saturday, 23 February 2019

Giving nice feedback

For a work environment to be successful, there needs to be trust among the people working there. Scrum echoes this in the main rule of the retrospective - everyone did the best that they could with the resources they had available at the time. Blame culture is being stamped out by progressive companies. People want to be able to approach senior management without fear (and senior managers want to be seen as humans). Everyone should feel comfortable in their workplace at all times.

All of this is a really good thing, of course. But it seems to me we are also in danger losing something important when it comes to personal feedback. Individual development often involves confronting uncomfortable truths about ourselves before deciding what to do about them, and a part of doing that is knowing what to look at. I've had too many conversations recently with people complaining that they don't get critical feedback. They know they are doing a good job - they are getting plenty of positive feedback - but they know they aren't perfect and want to know what to look at to develop.

So are we being too nice? If we don't give critical feedback when asked, is it because we don't see a problem? We don't want to upset people? Or is it we want to spare ourselves an awkward conversation? Speaking for myself, I very deliberately lean into being supportive over critical (yay me) but if I'm honest I can also see some of the last in my behaviour. Since people have been asking for more feedback, I've tried to change my own behaviour and be more open and offer more (constructive) criticism.

Initial findings are good. Firstly, I've had very positive feedback from people who've been pleased to have something pointed out. I've seen pennies drop and cogs start turning in people's brains. It feels like I'm helping people think, which is the object of feedback regardless of the context. Secondly, it has helped my own development. The first time I gave some properly honest feedback I could feel my own fear and anxiety welling up. But the person wasn't upset or horrified and they still talk to me. Over time, having awkward conversations has gotten a lot easier - and this is a really important skill to develop when moving into management. I'm really grateful to this person for making me think a bit differently about all this and one day I might find the courage to tell her.

Clearly, honest feedback is something to be given sensitively. Although the above might read like it, I haven't been walking round giving the benefits of my "wisdom" to anyone unlucky enough to be within shouting distance. I'm still very careful about who I talk to and when, and it's only people who are asking for it and I trust to have a sensible conversation.

How feedback is delivered is also important. For me, the classic "shit sandwich" technique misses the point somewhat here. We are not talking about problems, we are talking about development opportunities. We are not looking backwards at everything that was "bad", we're acknowledging the past and looking forward at how things could be in the future.

It's also important to ground the conversation in a concrete reality. That means talking about the impact - some of which might not have been noticed at all. For instance, "you are very smart, but are naturally quite blunt" is feedback but invites a "so what?" response. "You are very smart, but are naturally quite blunt and this means people who work with you can find you intimidating" is rather more significant. It speaks to a real (unintended) problem and also leads to a useful discussion about communicating ideas, bringing people along, working with people from other disciplines, and so on. This is all good stuff and it also gives the person enough to kick-start some positive self-analysis.

Of course all this must come with the caveat that it's aimed at helping good people be better. It's important to work with each individual in a way that suits them.

What's the conclusion here? Giving feedback is important, and I think it's very easy to make it meaningless by avoiding the difficult conversation. This does a disservice to those around us. Being too nice doesn't help anyone grow.

Saturday, 19 January 2019

Tagging documents in Google Drive

I use the Google suite for various types of writing, including D&D. Google Drive will let me organise files by folder, which is very early 1990s. What I'd like is some kind of tagging system, so I can find the docs I have scattered all over the place. While it seems like something Google should probably support, while we are waiting for a miracle I can roll my own system using Google Scripts which is apparently mostly ES3, with some enhancement from ES5. So that's ... late 1990s. I guess that's an improvement?

Anyway, language hell is ahead of me and I'm sure at some point I'll convert it to use a proper API in a proper language. Right now, I need a simple way to do tags. Since I don't use the description metadata field (or indeed really acknowledge its existence until I started this) I think I'll use that. Adding something like #npc #story to the field is the level of work I'm willing to do - what I want is something that will pull these tags and list my files using them.

First things first - how on earth do I start writing Google Scripts? Obviously as a lovingly maintained part of the Google suite it'll be tightly integrated so all I have to do is ... add what it claims is a plugin to Drive and I can start creating script files which open in an in-browser editor (I hate in-browser editors, but here we are). Then I need an index file and something to call it. It seems that when loading, the doGet method in the file is called and that needs to hit the HTML file. This all needs accessing via a URL to the dev or published version. Ok:

Carefully not thinking about what will happen if a user isn't using Javascript, let us push forwards. Templating seems pretty simple so an HTML file can be put together. I can also drag in a CSS file from somewhere else so I can reuse my awesome styling.

All I'm looking to do here is create some lists, so I just need to get the filenames and links and put them on the screen. Specifically, I'm getting a list of tags then creating a list of files per tag.

Google Scripts provides a load of objects to poke and since it's taking care of the authentication (the main reason I'm doing it this way rather than using the api) this should be pretty straightforward. I mean the only way this could go wrong is if the scripting is using Javascript from before the dawn of time and I have to do absolutely everything myself?

To be fair it's not too bad once I've got my head out of Ruby and back into Javascript and then removed all the ES6 goodness I've come to expect in the years since it became commonplace. I can actually craft exactly what I'm looking for:

Under the hood ... eugh. It's pretty ugly - and not just because I've used two different forms of a forEach loop in a single short script. To generate this page, I'm looking at each file to get a list of unique tags then reversing the process - looking at each file to figure out if it has the tag I'm interested in. This is not Efficient CodeTM. On the other hand, it is Working CodeTM so it'll do as a proof of concept. I'll make it less awful when it gets too slow or I translate it all over to the API.

In the meantime, I have a document tagging system, and I can still use my favourite online text editor, even if I had to write the code in my less than favourite online code editor. And here is the code all in one place. One day I'll get tagging and Markdown support from Google itself...

Monday, 31 December 2018

The year that was, 2018

2018 is more or less over and it's time to look back and decide if it was worth the bother. Work has been tough this year, with a new role and responsibilities but despite that I've managed to improve my physical and mental health (especially the former). But it's not good enough, and I think 2019 needs to be a year where I focus on sorting out my personal life. I'm still not creating enough, and that is in part because of the limited time I spend doing useful things outside of work. Still, this is post is about the things I have done so let's look at the positives. It's not all Elite Dangerous and YouTube.

  • 13 posts on this blog (14 including this one)
  • 2 posts on my gaming blog
  • another year of the Year in Pictures site, up to 21 photographers
  • created the Year in Pictures helper and only completely broke it once
  • over 239 contributions to various projects on github (down from last year)
  • started a new D&D game (Elsewhere) resulting in much writing, drawing of maps and running of sessions
  • went on another photography course to learn how the rest of the buttons on my camera work
  • 100+ pictures on my instagram account
  • loads of photos on my flickr stream
  • became a Reader at the British Library (and failed to find the thing I was looking for)
  • helped out at Codebar
  • lost a load of weight
  • started back at the gym properly
  • updated my website and Gareth's website
  • made a set of coasters with pictures of my nephew at different points over his first year

It's not a lot, but this has been a very work-centric year. That is definitely something to change in 2019. On top of this, I've also learned to pick a lock, introduced some friends to the joys of the escape room, done a load more sound desk stuff, and failed to see the friends I mentioned last year.

Resolution count: 6/10 - definitely improving over previous years.

This coming year needs to do more with what I started this year. More serious exercise, with an endgame I'm actually happy with. Code more. Draw more. Create some things that aren't the blog / game cycle. It's going to be another busy one at work, which means I need to be careful to avoid just getting sucked into it. After a scientific analysis of work and play I'm going to be doing at least one thing every month. And I will definitely see those people this year.

Brace yourself, everyone. 2019 incoming...

Saturday, 29 December 2018

Mental health and maths

There are a million connections between social media and mental health and most I know about show a strong (albeit anecdotal) negative connection between the two. People are constantly saying they've taken a break from Twitter and feel much better for it, and I know that for me avoiding "debating" (or more accurately "screaming impotently into the void") on social media is definitely a good thing.

But what happens if we look at this backwards? Engaging with the world, as opposed to withdrawing and shutting down is supposed to be positive, right? In theory, social media is about reaching out and connecting (haha I know, but stay with me). I know that when I'm feeling happy and on top of the world I look at it with interest and very often take photographs of the things I see. Some of those photographs end up on Instagram. When I'm exhausted and my mental health is poor I shut down to preserve energy and I feel like I'm doing a lot less, such as taking fewer photographs.

So in theory I should be able to see a correlation between taking lots of photos and posting them to Instagram and times of good mental health. Good - I've needed an excuse to write some code. Cue the music.

A bit of hacking later and I've got a bit of Ruby which looks at my Dropbox Camera Uploads directory and counts the photos I've taken this year, grouping by month. And I've got another bit of code which looks at my Instagram feed and does the same, but using a deprecated API because I'm not a business and apparently that means I'm out of luck in their new world.

And lo:
The very obvious peak in October is at a time when I had a week off, had just had some very good news and actually Did Things outside of work (walked the Two Tunnels walk in Bath and went to a display of Japanese art in Bristol). Coincidentally I also wrote a post about having a peaceful time off. So it was a good month - or at least a good week in a month.

At the other end of fun, the two obvious troughs are in July and November. This matches my memory of the year - times when I had to drag myself out of a major slump. This was especially true in November, when I had far too much to do both in and out of work. In work a long series of things went wrong and I was on the back foot the entire time - which resulted in me writing a post about reactive vs proactive work. Outside of work I basically stopped doing anything as I worked late and spent the evenings recovering. I felt my engagement with the world around me suffered and this is born out in the numbers.

Otherwise it's pretty stable through the year other than an upwards tick in August (another holiday), so what happens if we add a trend line?
An increase in photos taken over the course of the year, but interestingly a decrease in photos posted to Instagram. So I'm snapping more, but gradually getting less interested in the world around me? That's quite possible. I mean, have you seen the world?

Last year I had an interesting time and I started this year writing about recovering from an actual crash. So what happens if we overlay 2017 Instagram engagement with 2018 engagement?

*hack hack*

This happens:
I was at my lowest in November and early December 2017, which correlates nicely with ... the highest engagement of the year. Hmm - that messes with my theory. December I can easily explain - I was away from work for most of that month, actively recovering, and spent Christmas with my parents walking miles and taking photos of their village. November is a mystery. Looking back, it seems in November I went for a few long walks around London and posted multiple pictures from each which brought the count up. Confusing. Moving on to trends.
Rate of posting to Instagram increased over 2017, which is mostly expected with the exceptions above. The decline in 2018 still worries me, as does the change in style of my pictures over the last 24 months. I used to put more thought into use of abstract space and filters, giving more texture to the posts. More recent photos are pictures with limited work on them - pictures of things rather than more artistic expressions, if you'll forgive the phrasing. It's not all bad - the technical quality of my pictures has definitely improved - however it plays to the narrative of less engagement from me in the output.

So what have I learned? This kinda works. It's certainly not good enough to create an early-warning system to tell me to Take Steps if my frequency of posting drops below a certain level. But there are some interesting patterns and going through this has got me thinking about changes I can make in life which will hopefully make a positive impact. I have strong, positive memories of the various Things I've done, and those Things tend to generate photographs. So if, loosely, I can do more Things I should improve my general outlook on life. I hope? I spend a lot of time in London, so finding something interesting and photo-worthy to do once a month shouldn't be the hardest thing in the world. Now I need an arbitrary point in the year at which to make fresh-start promises to myself. Some kind of Resolution for a New Year, perhaps?

If you're interested in details, the code is all on Github and  I used Google Sheets for the spreadsheet to generate the graphs. I am not a doctor and I'm well aware this is not Science.