The year that was, 2023

Well, it's the end of another year and it has been a strange one. It hasn't been a terrible year, like some recently. But I also don't seem to have done anything except work. In review at the end of the year, I feel I've learned a lot and I feel very tired. So let's see whether I've achieved anything at all - there has to be a bit more then Hell Let Loose and Roboquest.

• 14 posts on this blog (15 including this one) - ninth consecutive year of a post per month
• Started posting my blog posts on LinkedIn - small thing, but a big deal for me
• Another year of the Year in Pictures - ninth year and we're up to 40 photographers
• 224 Github contributions - well down on last year
• Rebuilt my email setup so email actually arrives
• Finished replatforming all my apps
• Started playing D&D again
• Started writing a D&D setting too
• Some reading
• Managed to see a few friends for the first time in years
• Foraging cookery course
• Completed another escape room
• Stripped down and repaired the taps in my kitchen
• Finally sorted out the signatories on the house bank account (2+ years!!)
• Another year of exercise with a personal trainer
• Established a good physiotherapy regime
• Another year of the Saturday morning exercise class
• Won an award for CIO potential
• Spoke at a conference about the future of tech
• Yet more instagramming
• More photos on my Flickr stream

Resolution count - 3/10. Utterly awful.

This time last year I wrote this:

"Looking forward to 2023, I'm going to write the same thing I write every year. I need to spend less time working and more time living."

It really has been a strange year. I seem to have been constantly busy and yet outside of work achieved very little. I feel like I've spent so much of the year on the back foot - whether that's because I've been worn out, or because I've spent time doing things for other people and not moving forward with things I want to do. My resolution count shows this - lowest in years. For various reasons, I've not been able to prioritise much for myself. I've learned very little, and not completed many projects leaving an overall sense of emptiness.

It has been a sub-optimal year for physical health. I've had a year of blood tests for chronic fatigue (fortunately nothing) and all manner of problems with my hips which have blown apart my fitness. I've been nursing injuries since May and while training and physio has helped, I feel like I've moved backwards (and outwards) this year. This is something I really want to turn around in 2024 if I can avoid damaging myself any more.

Also, while I don't feel too bad as I write this, I think my mental health has taken a battering this year. My work / life balance is completely off - the worst it has been in years. However, unlike previous years when this has been the case at least I'm enjoying the challenge this time around. But that doesn't change that work has been full on, which has led to much exhaustion and doing less of my own things while I hid and recovered.

All that said, while my real-life has been less than ideal my work-life has been doing ok. I have achieved a lot in my current role and even won an industry award which was rather surprising. I don't write about work much during this year-end post but this year it's worth calling out the success to contrast ... well ... everything else.

So some words for 2024...

Looking forward to 2024, I'm going to write the same thing I write every year. I need to spend less time working and more time living.

Sigh.

More seriously, I want to focus on my health - physical and mental. I need to increase the quality of my time across the board. The time I spend at work needs to be better spent. Outside of work, I need to make better use of the time I have - doing more, and resting better. I definitely need to do more myself instead of chasing round after other people. This year - quality over quantity.

Time for 2024. Tally-ho.

Professional new year resolutions

It's my favourite time of the year - the null zone between Christmas and New Year where nobody is doing anything, the weather is awful and hiding away is socially acceptable. I can sit with a laptop, learning about things I never find time to touch during the year and slowly become one with a chair. All without any guilt that I should be doing something else.

Every year I write new year resolutions and then do an end of year summary however this always focuses on my personal life. This year has been VERY work-heavy so I thought I'd also write some extra new year resolutions for my professional life and why I think they are important. It's the holidays so don't expect anything enormously insightful - there is a huge list of things I should be doing, and this is more about where I should refocus myself for the new year. Here we go.

More engagement

Anyone who has read Future Engage Deliver knows that you can have a vision (Future) and set everything up for output (Deliver), but if you don't bring people along (Engage) your success is going to be limited at best. This ranges from simple visibility through to detailed talking about strategy to everyone, both in the team and wider. When physically in the office some of this happens just by not hiding behind a closed door, but when working remote those corridor conversations don't happen - there is the need for more deliberate mapping of communications and touchpoints.

My area is now essentially a remote working group and that creates a need for more deliberate communication. We're also in a time of change - a very exciting time, but change is always unsettling. This is going to make engagement even more important.

I don't think I'm too bad at engagement - but only when I prioritise it. There has been a LOT going on, and good engagement takes time and energy - two things that have been especially in short supply in Q4 2023. So for 2024 I'm going to look to make two changes. First, make sure my engagement is of a higher quality. Second, and more important at the moment, I'm going to prioritise engagement which means finding or making more time for it.

Give back more

The Tech industry can be a pretty bleak place at times, and the only way that will improve is by all of us doing what we can to make it better. Over the years I've tutored, coached, mentored and set up groups. These days I'm still mentoring people across several different organisations - particularly people who are leading software engineers, and software people who want to progress into senior management. I also sit on the London Data Board, I've done some consulting with groups looking to restructure (both paid and pro bono) and I've spoken at conferences about technical leadership.

I get a lot out of these kind of things and I am going to assume it's helpful! I'd like to find ways to do more. If you'd like a chat do reach out. Otherwise, I need to explore some ideas around this - ways to share / cooperate / mentor and so on. I'm not very clear what I can do beyond more mentoring (I'll post about that in the new year) but I'd like to find something.

Do less

Both of the above will take time and while we insist on saying "make time" this is actually impossible. Time is a fixed resource and needs to be spent effectively. What I want to do in 2024 is to make sure the time I have is spent on quality activities. For 2024, that is going to mean two things. First, a renewed focus on delivery - getting things out the door rather than spinning wheels and sustaining. This will also help with the good feels - everyone likes achieving things. 

Second, I'm going to ensure the various meetings and governance processes I attend are providing proper value. As mentioned earlier, we are in a time of change and that can easily mean overlapping or redundant org processes. This only costs us time and energy, so finding and stopping them helps not just me but everyone. I'm going to push back hard where it seems appropriate to do so to protect my time.

Work / life balance

Actually, there IS a way to "make time" - work more hours. That has been my default in 2023 and to everyone's surprise it has proven unsustainable. In 2024, I need to balance work and life better. This doesn't just mean looking at working hours - I need to put things in my off-time and have the energy to do them. I know there are problems when I'm not doing anything in my free time because I'm too tired so this is a move to protect my mental health.

So that's it. In 2024 I'm going to work on how I engage with people, I'm going to find ways to give more back to the industry, I'm going to focus my time at work and I'm going to improve my work / life balance. Any guesses how I'm going to get on? Happy new year!

Sending email in 2023

"Your email keeps going into my junk box" - everyone.

I use a vanity domain to front my email address. I used to run a simple setup where the domain was basically masking my Gmail account. Incoming was handled by a wildcard forward in the domain host. Outgoing, I simply rewrote the email envelope with my desired email address. Essentially I was spoofing the outgoing email.

Gmail used to let me do this, but clamped down years ago requiring proper authentication with an SMTP host however the old setup still worked, as long as I didn't change anything.

Then the big email providers started clamping down on this kind of thing. In an effort to combat spam, email is increasingly complicated and the wider ecosystem is getting more locked down. There is a big rumble about the big providers essentially pushing smaller email providers out by blanket not trusting them, making it increasingly difficult to run your own email setup. This post is not about that, rather it's how I stopped my email started going into junk boxes. I was forging my own sender address, which is exactly the kind of behaviour you see from various types of spam. Nice.

So, on the assumption I wanted my email to arrive I needed to revisit my configuration and set this up properly. I did a bit of work, so I thought I'd write up here so I can repair it in future if needs be, and it's in one place on the offchance it helps anyone else.

Incoming email - you're emailing me

Not many changes here - although I use a combination of Cloudflare and Ionos DNS these days, but a blanket forwarding rule in the Ionos config from the whole domain still works.

Outgoing email - I'm emailing you

Ok, this is where it gets interesting. I can still send email, setting the domain to whatever I want, but my emails are being flagged as spam. This is because the receiving hosts are trying to protect the account owners from spam and my setup was being flagged as spam. Obvious note - I set up a test Gmail account for receiving email so I could test the effects of my settings.

Outgoing SMTP server

First thing was properly configuring an outgoing mail server. In theory, this can be done with the Gmail SMTP service but while I could authenticate properly I found my email still ended up flagged as spam. I'm sure there is a way to do this properly but for the moment I instead turned to SendGrid and this documentation was useful.

A free account allows 100 emails per day - plenty for me. Nobody wants to hear more of me than that. In the SendGrid interface it is easy to create a API key (Settings -> API keys) with appropriate emailing sending permission then when adding the server details, just select smtp.sendgrid.net / apikey / $YourKey. Only slight gotcha is making sure you get the port right (SSL over port 465). This should authenticate properly and email can be sent - although it'll probably be going to junk again.

Next up, setting up DKIM. This stands for DomainKeys Identified Mail - an email authentication method designed which allows the recipient to check that an email came from the domain it claims, and was allowed by the domain owner. The setup is found in Settings -> Sender Authentication. You might be able to get away with Single Sender Verification, but I did the full Domain Authentication. You need to be able to modify your domain's DNS settings for this to work properly.

If the setup doesn't seem to be working properly you can test the individual additions on the command line with a tool like dig.

dig foo8908.tomnatt.com should give a NOERROR response. If it's not, the setting isn't right or it hasn't refreshed yet.

Finally, assuming this is for personal email you'll want to disable link tracking. This rewrites links in your email for marketing purposes and likely break any links you send unless you configure it properly. Turn it off with Settings -> Tracking -> Click tracking -> disable and links will work again.

Other DNS setup

There are two other DNS entries that can help with proving email provenance - SPF and DMARC. I'm not sure whether I needed all these for a minimal setup, but they do work best when all three are present. I did configure them, so I'm capturing what I did. 

SPF (Sender Policy Framework) is another way to ensure the mail server sending an email is allowed to send via this domain. It works by defining which servers can send email, so the client can check, rather than directly encrypting the connection (the DKIM approach). The setup is fairly simple, and can be checked with tools like this.

An SPF policy which allows sending from Gmail and SendGrid servers might look like this:

v=spf1 include:sendgrid.net include:gmail.com ~all

DMARC (Domain-based Message Authentication, Reporting & Conformance) helps receiving mail systems decide what to do with incoming mail that fails validation via SPF or DKIM. So this is worthless without at least one of the other two.

A rule which tells the receiver to mark failing email as spam and send reports to the given email address would look like this:

v=DMARC1;p=quarantine;pct=100;rua=mailto:postmaster@tomnatt.com

Done

And lo, email appears to be flowing again. I hope something here helps. To finish, I want to note that I'm not an email expert - not even close. If you are, and you're seeing somewhere I've written something stupid please reach out and I'll correct and attribute.

A few words for World Mental Health Day

Earlier in the week (Tuesday 10th October) was World Mental Health Day. As part of normalising mental health through openly talking about it, at work I posted something of my own experiences on our internal chat board. The reception was extremely positive - people pleased someone spoke out, feeling seen and so on. So - since mental health isn't just about last Tuesday - I'm reproducing a version of that below, albeit with the work-specifics removed.

For World Mental Health Day I wanted to share something personal. Mental health still has a social stigma attached, and while this is improving rapidly even from five years ago there is a long way to go. So, to make my small contribution, I'm going to tell you a little about myself.

I have suffered from depression throughout my adult life. I'm fortunate enough that this has very rarely reached the more intense levels where I can't get out of bed or worse, but I've certainly had ongoing difficulties with anxiety and feelings of emptiness, loss of interest in life in general and feelings of hopelessness, guilt, worthlessness, helplessness and others. Even writing this makes me feel like a fraud, since it's not "proper" depression etc etc...

I know that these feelings go in cycles and I've spent a lot of time identifying my own triggers so I know when I need to take a step back and protect myself before I get into a worse state. I also know that it sometimes sneaks up, and I suddenly find myself in a bad way. It's hard and deeply unpleasant at times.

I'm sharing this mostly because I don't think people talk about mental health openly enough, and World Mental Health Day seems like a good day to put something out there. For some people, I hope to show some solidarity. For others, this will be some small insight into a world that is totally alien to you.

On this blog, sometimes I post about mental health. If you'd like to see what I wrote after taking about a month off you can see that here, and you can see all my posts about mental health here. Obvious disclaimer - I am not a medical professional.

Most important - if you're in a similar position, know you're not alone. There are people who share your experiences and are willing to talk. If anyone would like to talk to me, please feel free to reach out. I'm also quite happy for the "but what's it like" conversation if you'd like to ask that.

At work, I suggested a discussion group and it occurs to me that that might be a useful thought in other contexts - either for people who have different experiences they want to share, or for a wider group to talk. If people think that is a good idea do shout and if enough people are interested I'll see if I can get something set up.

Take care everyone.

Some thoughts about the future of the Tech industry

Last week I was given the opportunity to sit on a panel of technical leaders and talk about the future of technology. I had a few notes about about how we're going to need to change our thinking about building capability and I thought I may as well capture and flesh out a touch the results of my crystal ball gazing here.

I spoke briefly about three areas:

• The people we hire
• The expectations of our users, and our expectations of them
• Where I think we’re going to need to invest and build capability

The people

Fairly obviously, Technology is getting more important to daily operations. But it's getting harder to hire people all the time. As we all keep hunting for talent, those who aren't offering the top end salaries will increasingly have to look nationally or even globally to recruit. I don't think moving the organisation to another city is a sustainable approach - at best, it will simply move the problem. Instead, I think we will increasingly see a more distributed workforce, and therefore more remote working. As staff turnover is identified as a major organisational cost, we’ll also see more emphasis staff retention - succession, training, individual growth and so on.

Whoever nails building a strong remote working culture and environment which encourages loyalty and celebrates and develops the individual is going to do very well. I think the secret to this is going to be building very strong communities of practice, and if I’m right we’ll see more “Head of Community” style work and roles growing up. 

I also think we’ll see more Tech decisions based not on the best technical or product solution for the org, but the best fit for the skills we can grow or hire. This would suggest a lean towards the big names (the Microsofts, Googles and Amazons) who are heavily investing in training the Tech industry through free access to courses, sponsoring hackathons, and so on.

I believe this will be more acute as Automation is used to deliver more while avoiding the continual growth of IT departments. We will need to retain the skillsets to maintain that automation layer or we'll be seeing yet another wave of technical debt.

Expectations from and on users changing

In times of yore, users used to have to know something about operating a computer to install and use software. These days, users on a smart device can just touch an icon and get everything they need. This is a victory for accessibility and digital inclusion but it also means the gap between “technical” and “non-technical” user is widening. Our helpdesks and other support points will need to work with an increasingly broad ranges of questions, especially as tech like AI gains traction, and expectations for what it can do are all over the place. 

We’ve also been seeing for years the expectations from users increasing as they use more SaaS products at home and demand the same sort of tools at work. To satisfy these needs, the cost of development is going to go up and cover a wider range of skillsets - and of course this links back to the earlier points about skill availability. Out of the box services are also going to be affected - vanilla deployment is going to be less palatable in the office, requiring more work for a good result especially with the current state of many internal systems user interfaces with respect to accessibility and usability. 

This is particularly true regarding what have often been considered secondary requirements - accessibility and environmental sustainability for example. Users are (quite rightly!) far more vocal about accessibility needs, and we need to not just respond but get ahead of their requirements.

Other places we’re going to need to build capability

Technology is obviously an increasingly essential part of everything. I mentioned the effect on helpdesks above. We're also already seeing increasing amounts of security threats and the wider reaching impact of a successful attack. This will take us into an ever more expensive arms race in the Security field, which will mean building Security capability. This is going to need to be approached very carefully as it will be very expensive - everything I’ve said about skills shortages are far more acute in the world of InfoSec. Part of the Security picture is a renewed emphasis on good, basic engineering practice (such as patching) - but again, this places a challenge on building skills in our organisations.

We’re also generating and handling more data all the time, so inevitably we’ll see more human error leading to data loss. In fact, for any organisation a major security incident or data breach is only a matter of time now. If we are assuming that it is going to happen, there is a need for much more robust organisational responses to these scenarios which means building appropriate incident response and Business Continuity capabilities. Of course, just responding isn't enough so there will also need to be stronger data ownership throughout our organisations, with more people with data owner and controller roles. Organisations will need to fully grip their end to end processes and user journeys in ways that perhaps hasn’t been happening before.

Obviously there is a lot more that can be said about everything here!

Looking to the future

One of the symptoms of seniority that I still struggle to process is this idea that people want to hear what I have to say and understand how they can (professionally at least) be me. It's a weird feeling, both humbling and frightening, and has only grown in the last few months. I'm being asked to speak at conferences, and in a few weeks I'm on a panel discussing the future of the Tech industry. Why? Surely there is someone better for this?

Honestly, this is a question I've been asking myself since my time in the Civil Service, where I found myself in a meeting discussing how to spend five million pounds of public money. I should write a post about getting over myself and getting on with it. This, however, is not that post.

I moved away from development into management so I could make a difference to the industry. How do people get into Tech? How do people move to more senior technical roles? Showing the importance of senior leadership with technical backgrounds, and helping people get the right experience to be credible candidates. These are core to my motivation and it seems I have the beginning of a platform. This is a responsibility I want to take seriously. What can I offer to the wider industry?

I've got a few ideas and since there seems to be a few people reading this blog from LinkedIn, I suppose I can offer them up. First and foremost, I'm thinking of creating a set of interviews with people in my position telling their story, talking about how they got to where they are today. I don't kid myself that I'm some kind of Steve Jobs or Bill Gates, but then neither are the overwhelming majority of people. I am, however, fairly successful in the industry. I'm in a position that someone getting into Tech can reasonably aspire to attain. Not inspirational perhaps, but practical. There are lots of people like me, and the stories don't get told often enough so I like the idea of creating some kind of resource showing the different paths people can take. At the moment I'm thinking a podcast, but format to be decided. If you're reading this and think it's an interesting or useful idea, please do reach out.

In addition, I sit on the London Data Board, and I'm happy to offer some time to support other initiatives if they come up. Again, do reach out - especially if they are something to do with nature or animals. On top of this, I already do some mentoring, and I've got a whole post about that coming up soon.

I opened saying that people asking about my career and how to "be me" is weird. Actually, on reflection, it's a weirder feeling that I'm in a position to be feeling a sense of responsibility to the industry and asking any of the above questions. It's a great privilege to be working somewhere where I can spend some time looking outwards and hopefully make a difference. That's my intention anyway - so I've written it down in public so I can hold myself to account.

A break in the routine

When I started as Director of Engineering at Macmillan, one of my objectives was to build a department which functioned without me. I wanted to be able to pull myself out of the operational day to day, empowering others to own those problems, while I focused on overall direction and wider concerns. This was always an aim rather than a hard goal - similar to the sysadmin end game of a fully automated system, it's something I would like to chase and I think I can get close, but I don't believe I will ever truly achieve.

Over the last year and a bit I've set things up to work this way. I've created core areas of my division, and put someone in charge of each of those spaces. I've empowered them to make decisions and own their successes, and I've carefully worked with each of these leaders to identify and remove weaknesses. There is still a long way to go, but I think we've made a strong start.

Why write about this? Well, I took last week off as leave (booked some time ago) and at the end of the week:

The Boss: How are you doing?

Me: Starting to feel better! After about four days asleep, I finally started to feel human again.

The Boss: Want another week off?

Me: Is this some kind of trap?

It wasn't a trap, so after a quick conversation I decided against looking that gift horse in the mouth and spent half an hour with my assistant going through my diary. The first week off was all planned, so that hadn't been a problem but this second week was a spur-of-the-moment thing. Surely it was going to cause all kinds of problems if I wasn't around?

Nope.

Of the week, there was an hour which absolutely required me and needed rescheduling. Which means ... what? This is the question I've been pondering for a few days. On the one hand, my ground-in desire to be needed and adored took a massive hit. Being basically superfluous is not good for the ego. On the other hand, this is exactly what I've been building towards. The department can survive without me. It can make decisions, move forward, solve problems, etc etc...

We've got a way to go. The wheels most certainly will not stay on the wagon indefinitely and the more forward-planning functions haven't been tested in this short term. Plus, of course, people are covering  some bits of my role and I shouldn't underplay their help. There is still plenty of need for someone in my position to keep things steady and move things forward - I don't think they can make me redundant just yet. However, I'm pretty pleased with the results of this (accidental) test and what it means for changing my day to day when I'm back.

To be clear - I'm not wasting my time. The bits that didn't need rescheduling were parts of the job where I'm providing support or oversight, and direct support for individuals. All this should be done, but I can certainly ask some questions about what happens if I dial it back a bit. That means less oversight, but perhaps we're ready for that? Then I can focus more on the work I'd be doing myself which isn't getting done. However, the nature of the job is such that my work is (supposed to be) much more strategic which naturally has a longer burn. If it's delayed a week, that doesn't create the same kind of problem as operational work stopping. Fortunately, this also means focusing in on the part of the job I particularly enjoy.

So, in summary I am very deliberately in the process of making myself operationally irrelevant and it's time to reboot my focus list. Is this ... is this what success looks like?

Next CIO 2023

Well, this month I won an award.

Honestly I'm not sure how to write this post. I do not usually deal with praise well, and that is unlikely to change. However an industry award is something to celebrate so I'm going write something about being recognised, and I'm going to keep self-deprecating jokes to a minimum.

Next CIO is an industry award where a group of expect-to-be-CIOs-soon are recognised as the future of the industry and celebrated. They are brought together over the year for workshops and group learning, and hooked up with a mentor - someone who is already a successful CIO.

And I am one of these people. Which means I'm part of the future of our industry. Good grief.

What does this mean? Well, over the next year I'm looking forward to talking with and being mentored by the very generous Wayne Clements, and I'm going to be attending a series of workshops with my Next CIO cohort. I've also got access to a group of excellent people from across the world of Technology - there has to be something useful I can do with that set of contacts.

For the award event, we were asked for volunteers to talk for a couple of minutes about ourselves. I volunteered mostly because while I didn't want to do it, I wanted to make sure I was making the most of the opportunities. That is what this year is going to be about for me - taking opportunities and seeing what happens. In this case, what happened was a talk which got a few laughs (in the right places!) and some excellent conversations in the bar afterwards with people who were interested in what I had said. Only good things.

Looking forward, I'm going to do a few things. I'm going to start posting my blog posts on LinkedIn. Long-term readers will know I write this mostly as a tool for self-reflection. Putting them up on Twitter and Facebook was a big step and LinkedIn will be another (slightly terrifying) moment of development. So ... hello if you're reading from LinkedIn. There are lots of other posts here - you might be interested in the essay subset.

Coming back to the topic of my "about me" talk - I need to think again about how I give back. My particular interests are around how people get into the Tech industry and how they move around and up. Over the years I've coached and mentored, and I want to lean back into this. However, this doesn't scale terribly well - there are only so many hours in the day so there is only so much time I can give - so I want to look at other options. There are a lot of stories out there to be told, and while some people have inspirational tales of mega-success, I think there is a lot of value in the more attainable stories of people who do well without being the next Jeff Bezos or whoever. Can we get better access to those tales?

Anyway, these are thoughts for the future. For the immediate now, I'll end with some pictures from the event and me talking about how I should have been a forester.

Making a decision

I've been thinking a lot about governance recently. In general, I like consensus - it's important that expert voices are heard and action is taken on the back of expertise. I dislike dictating how things are done for the same reason. Generally, I find people thrive when given the space to take ownership of problems and work through them.

However. This can create a few problems. The first is mediocrity - if everyone is compromising to form consensus then it is far too easy for nothing to actually be good. In the worst case, the important part of a proposal is compromised away and any outcome becomes a waste of time. This assumes there IS an outcome, which leads to the second problem - how is a decision actually reached? How do I stop people talking forever?

This latter problem is the one that concerns me more at the moment. I see different versions of non-decisions all over the place, from email threads which ambiguously leave "someone" to do the next thing to conversations which always need another person to engage before maybe everyone agrees. At worst, this never concludes but even at best it is sloooooow.

For me, this leads directly to the question "what is a decision?". When is something approved? When is it agreed? Or when it is simply some positive noises coming out of a discussion? It's important to draw this out for everyone involved. Individuals do not like being misquoted ("no, I didn't actually agree to this"), project managers deserve to know where they stand with sign-off so they can move on ("is that a decision?"), and I need to add some formality so I don't have to mediate these positions.

I'm leaning into some hypotheticals here - while I see some of the above at work, decisions are certainly being made, and people are thriving in their empowerment. However, and being selfish for a moment, it is actually my needs that are being neglected. As my role changes (and reflecting on my musings from my time covering as CIO) I find myself with less and less time to mediate circular conversations. So while things generally work, I need to move this from "generally" to "always" and "with less effort from me".

It's time to make some changes. As noted above, I've previously been happy to give people space to feel things out and learn from them. I don't want to lose that - it's an important part of empowerment and learning. However that is the discussion part - I need to add some stronger gateway decisions to the way we work. On reflection, I think I need to be better at separating discussion from decision and creating a formal record. There is too much going on at the moment to be imprecise about the state of decisions, so I need more opportunity to eyeball people and tell them to speak up or forever hold their peace.

The trick through all this will be retaining agile working patterns - I don't want to create some kind of decision auditing circus, but as we mature we both want to and need to be able to hold each other to account properly. This means having something to be held to - robust decision records where people can make their decisions known formally.

Initial muse for the moment - I've been exhausted this month. More on this later I feel.

Slack notifications from Github Actions

A while back I wrote about moving my deployment scripts away from Codeship into Github Actions. This has continued to work, but as I noted it did leave me without notifications in Slack. Time to fix this.

There are a few ways of enabling notifications. The best way involves creating Slack apps with incoming webhooks, but this involves a lot of faff (especially on a free Slack instance). The ever-useful Phil Wilson found a much simpler alternative.

Enter the Github Slack app.

The setup is very easy:

• Add Github app to Slack
• Log in to Github app as whichever user should be receiving notifications
• Authorise it to talk to your Github account in Github

All three of these are documented in the above link.

Then it's a case of setting up the notifications you want within the Slack app channel. I used the following:

/github unsubscribe $account/$repo issues pulls commits
/github subscribe $account/$repo workflows:{event:"push"}

The first removed the overly noisy notifications from a whole bunch of Things on the account. The second enables notifications from workflows (ie Actions) - mine are triggered on "push" events. Docs for the workflow configuration can be found on the integration page.

And behold! Notifications!

When Github updates its SSH host key

I wrote recently about deploying to remote servers via Github Actions. I use this blog partially as a place to write bits of useful documentation and this is a quick note on how to fix those deploys when Github updates its SSH host key. It took me an evening of fiddling around while watching TV to get this right, so I don't want to have to think about it next time.

On the target server, remove the old github.com entries and add the new key into known_hosts:

ssh-keygen -R github.com
curl -L https://api.github.com/meta | jq -r '.ssh_keys | .[]' | sed -e 's/^/github.com /' >> ~/.ssh/known_hosts

And that should be enough to make it all work again.

These commands courtesy of the Github blog. Essentially, this post is distilling that one for future reference.

Hanging out with the cool kids

I have spent most of an incredibly busy March as a stand-in CIO while my boss was away. After three weeks of my first real taste of a C-level role and board member, I find myself grateful for the experience and thinking about all the things I've learned. Time for some reflections.

The job itself was obviously truncated - covering is an inherently tactical affair whereas a job like CIO is strategically focused. So I had an easy ride from that angle - large amounts did not need doing in the timeframe. What I DID do involved a lot of reporting and representation. Much more than my current role, I had to think about information flow across the department and that naturally led my mind straight to the way the processes were set up. Our department has been entirely overhauled over the last twelve or so months, and it was very interesting getting a different perspective on how this fits together and where we can work out kinks to prevent our own internal reporting industries wasting time re-collating information instead of letting it flow.

In fact, I find myself thinking about the process far more than the content. How do we get information from team to division to report to governance board while retaining its accuracy? Of course, the quality does matter - over the last few weeks I've fronted reports to the exec board, the spend group, the risk board and the directorate board to name just a few. There is no way I could have written all this myself, even with the information immediately to hand and that of course leads back to ensuring the processes are in place to create good submissions "automatically" with my input being editing and guiding the narrative of each submission.

Setting this up for success is utterly critical. I'm going to write a proper post on this sometime, but my feeling on my current job is that when one removes the fluff, I have two objectives - create clarity and make time to support / guide people. This was amplified again as CIO. I've already started updating the way we plan inside Engineering to more directly feed into some of the governance processes, removing some of the kinks mentioned above, and now The Boss is back I am going to spend more time working through this. I've been working on "create clarity" since day one, but I've got a different perspective to work from now.

So what else? Well, the flow described above creates more and more dependence on the people in the department. The more senior I get, the more I see this shift from doing to coordinating and this was a surprisingly large additional step. I thought I'd already shed all remnant of actually doing things myself, but apparently not. It re-highlights the importance of building a high-performing culture and all the questions therein - about trust, development, setting and maintaining standards, etc. These few weeks have also showed where I've been too tolerant - or perhaps more accurately where my tolerance will get me in trouble if I do a CIO role properly. In my current role, if something isn't quite done on time there is usually enough space for me to help them out or do bits myself. As the CIO, this is not possible. They have to perform and if I need to course-correct, I have to be able to explain and set them going with a minimum of my own input.

This creates additional pressure on me to always excel at communication. Always write clearly, speak clearly, show strong direction, etc. No ambiguity. It also requires more of others, and I found it interesting quite how acute I felt the need for clear and timely communication almost above getting the answers I actually wanted. One of the last things I wrote to the department was a gentle challenge to us all to think about how we can communicate better and cut down on noise.

It also emphasises something I've always told people who want to move into more senior roles. Know how to take ownership of a problem. The value to someone in a role like this of someone being able to remove a problem from your shoulders and make it all work without too much from you is phenomenal. Importantly, this is not just "offer support", it's do, own, lead. For me, it's the golden ticket to being noticed and given further opportunities. As it happens, it's also much better for the CV / interview.

Another aspect of the CIO job was the sheer volume of reading. I have to read a lot as a director, but the number of papers and (hhgggnnn) PowerPoint decks that need absorbing is ridiculous. I found myself reading them in the evening, when I could get a drink and be much more confident of a few quiet hours without any interruptions. This, of course, did wonders for my work-life balance. It highlighted three things to me. First, the importance of getting submissions in on time as the knock-on effect can be board members getting 130 pages of "excitement" to read for a meeting 48 hours away. This was ... not a highlight. Second, the importance of writing clearly and effectively and spelling correctly. Especially in Word, where mistakes are underlined. Get it right. Third, I'm going to have to develop my speed-reading skills.

However! There were some bits that were easier. While there was a huge amount to do, I found the immediacy of my normal job diminished. That is, there were many things I needed to look at but I could choose my approach much more and spend less time in meetings than I normally do. So while the volume was huge, I did find myself with a bit more time to breathe and consider. It was weird and I think this reflection is more that I need to change how I approach my day to day.

Also, it was very interesting seeing all the department machinery working away. There is a great quote from Yes Prime Minister (which I can't put my hand on right now, so you'll have to suffer my paraphrasing) where the PM is talking to his Private Secretary about the role and the PS is explaining that while there are many things the PM SHOULD do and any number of things the PM CAN do, there is actually very little the PM HAS to do. The CIO role felt like that - if I'd sat back and waved things onward I could have gotten away with doing very little for quite some time. So the old systems analyst part of my brain turns towards the problem of maximising that - which is the question "how do I build a department that doesn't need me". I actually mentioned this concept in the interview for my current job so it's nice to have it reinforced.

Not that everything was plain sailing of course. I had to intervene in a few projects to help them back on track. I had to front a few concerns and problems to the wider organisation, and internally. My favourite moment, however was in the dying moments. I told my colleagues I was logging off for the weekend, and thanked them for all their support. I had one last quick look at my email. Then the website blew up. P1 incident, all the sirens going.

Who said technology doesn't have a sense of humour...

Artificial Empathy and The Role of Emotional Intelligence in Effective Management

As a manager, you're responsible for leading and motivating a team of individuals with different personalities, strengths, and weaknesses. One key trait that can help you succeed in this role is emotional intelligence.

Emotional intelligence, or EQ, refers to the ability to understand and manage one's own emotions, as well as the emotions of others. It includes skills such as empathy, self-awareness, self-regulation, and social skills. Here are some ways that emotional intelligence can help you be a more effective manager:

1. Build better relationships with your team. By demonstrating empathy and understanding towards your team members, you can create a sense of trust and psychological safety. This can lead to better communication, collaboration, and ultimately, better performance.
2. Manage conflict more effectively. Conflict is a natural part of any workplace, but as a manager, it's your job to resolve it in a constructive manner. By using your emotional intelligence skills, you can identify the root causes of the conflict, communicate effectively, and find solutions that work for everyone involved.
3. Make better decisions. By being self-aware and understanding your own biases and emotions, you can make more rational and objective decisions. Additionally, by understanding the emotions and perspectives of others, you can make decisions that take into account the needs and concerns of your team.
4. Motivate and inspire your team. A manager with high emotional intelligence can inspire and motivate their team members by understanding their individual needs, providing feedback that is tailored to their strengths and weaknesses, and creating a positive and supportive work environment.

In short, emotional intelligence is a crucial skill for effective management. By understanding and managing your own emotions, as well as the emotions of others, you can build better relationships, manage conflict more effectively, make better decisions, and motivate and inspire your team.

---

The above was written by this month's guest writer - ChatGPT. For those who don't know, ChatGPT is a large language model - that is an artificial intelligence which has been trained on a huge body of literature to effectively generate its own content. On a whim, I asked it to generate me some blog post titles for posts about management and one that came up was "The Role of Emotional Intelligence in Effective Management". I asked it to write that post and the unedited results are above. Not bad, eh? Guess we're all out of a job pretty soon.

Well...

There are a few reasons the answer is "no" which other, more expert, writers have gone through before. In very simplified terms, this kind of artificial intelligence works by guessing the next word in the sentence using probability from analysing a vast body of existing literature (the "large" part of the language model). It generates text using a variety of language rules and learns some context from interactions in the conversation (eg in my example I said "please write the eighth suggestion" and it knew what I meant) and this all results in something that looks like it can respond intelligently to your questions.

So, again very simply put if you say to ChatGPT "Twinkle twinkle" it will respond with "little star" - not because it understands, but because the vast majority of times when someone says "twinkle twinkle it is followed with "little star". This also creates a problem with accuracy - ChatGPT has no idea what it's saying but that won't stop it saying it very confidently[1].

Anyway, there are a load of fascinating technical and ethical questions here but I want to look back at the post it created for me. This one is high-level but you can keep asking "how do I ..." and drill deeper and the answers are pretty good. However, to me they all feel like they lack some substance and are hollow checklists and that raises an important question to me. I'm less interested in "is an AI drawing closer to doing my job?" rather "what is the point of my job if it feels like a very clever predictive text system can do it?" 

And to note - I have deliberately been asking questions about emotional intelligence here. This should be what separates people from computers. Does management just lack substance?

Sadly, the answer is often yes. If a manager learns how to lead from a book and follows the steps then they will become ManagerGPT as above, and will indeed be an efficient but hollow step in a reporting chain. I've certainly met and worked with managers like this - people who seem to have learned about being human from a distance and don't seem to be able to reconcile the recommendations with their own actions. They sometimes do quite well, but they are rarely recognised as good leaders by those who have to follow them.

So what did ChatGPT miss? There are no anecdotes in there - it's a cold checklist of suggestions with no emotional warmth or grounding. It can't follow through - it's all well and good having the step "learn the names of your report's children" but it wont actually go out and do that. These are the kind of thing that separates "leadership" from "authentic leadership".

Most importantly, AI cannot give the gift of its time because it has basically infinite time available to it. This means even when it is developed so it can fake the above (and it will be) we as humans will not respond well because for compliments to land we need to know it's more than mathematically calculated, or for attention to matter we need to know there is some kind of cost to the giver. For me, this is why we laugh at that scene in Demolition Man where the man is getting a pep-talk from an ATM (before being beaten up by Wesley Snipes), but that same dialogue would be more poignant if spoken by actual people.

In short, writing about emotional intelligence and empathy and actually developing them are quite different things and for the moment this is where humans can still add value. And, obviously, we should be making sure we actually DO that thing. All humans - even managers.

If you want to have a go at getting ChatGPT to do your job for you, you can find it on the OpenAI website. I certainly haven't used it to generate papers for the board (although if I did I'd absolutely list it as a co-author).

[1] This example comes from Simon Willison's excellent post about Bing and is originally drawn from Talking About Large Language Models by Murray Shanahan.

Migrate from Codeship to Github Actions for projects using Mina

Oh look - another free tool has decided to eliminate its free tier. My last post was about migrating away from Heroku as it torched its free offering and now Codeship has followed suit. In the past, I have written about using Codeship as a part of my toolchain for CI/CD but over recent years I have moved away from it to Github Actions for running linters and tests and using the Github integrations of the hosting providers themselves for deploying web apps. This means I've migrated many of my projects away from Codeship already, but there has been one significant holdout - my static sites.

Usual disclaimer - this is for a personal project, so I'm more interested in free solutions than high speed or resilience. Docs for myself, maybe someone will find this useful, etc etc

The problem

My static sites are generated using Jekyll, and deployed via Mina. Mina is a great tool, but for the purposes of this post it opens a connection over SSH and executes some commands, which require some environment variables.

I also need to migrate my "tests" - which are just building the site locally to make sure it's not broken - so I'll note that step here.

Migrate test build for a branch

Making the branches do a test build was simple - set up Ruby, get code, run the test code. All very similar (and easier) than doing this with Rails and wrapped up in a short Github Action.

Migrate deployment

Making the deploy script trigger on merge to the main branch was a little trickier (full example below).

First gotcha - when one merges a branch into main, the event that triggers is a "push" event. Knowing that, getting it to work is really easy but that took a bit of hunting. It also means that a push directly to main will trigger a deploy, if your main branch isn't protected.

Second gotcha - environment variables. I didn't want my server config in my public code, so I needed to set some environment variables in the settings of the repository. There are a few ways to do this in Github - I settled on using "Repository secrets" which are found in the project Settings -> Secrets and variables -> Actions then "New repository secret". They are accessed in the workflow file as ${{ secret.MY_SECRET }} and if you want to use them inside a script, you need to pass them through explicitly by setting the env for that script. Example below.

Third gotcha - SSH key. Mina opens a connection to a remote server, so it needs access to an SSH key. I created a public / private pair on the server and added the public key to the usual place, then put the private key in the repo secrets as above. Using this action allowed me to load the key into my workflow easily, and then I could use ssh-keyscan to push it into the container known hosts, allowing Mina to work properly. Again, example below. Shout out to this post for the pointers on the SSH key setup.

And lo, deployment works. Here is the complete deploy Action.

Tidy up

All that is left is to tidy up. Couple of simple steps here:

• Remove Codeship keys from server
• Delete Codeship project / build

And we're done! Like my previous Github Actions implementations, this is almost entirely generic so I can copy / paste into other projects with minimal effort and make sure I have light CI/CD for my fun projects. Now it would be great if I could get back to some of my creative programming projects for a while, instead of having to rebuild toolchains.

Migrating away from Heroku

Along with almost every developer I know, I was very sad to see the end of Heroku's free tier. I was hoping they'd reverse this decision, but it seems that was too optimistic so in the end I had to choose between killing all my toy applications, paying money or finding an alternative.

I did consider just shutting things down, but I had two concerns with this. First, I have one application I actually use and losing it would be a pain. Second, and more importantly, if I have no solution to this problem it adds another barrier to me coding as a hobby. So this needs solving, and not just by throwing hundreds of pounds a year at it.

Finding an alternative took a long while, primarily because I was trying to find a straight all in one alternative. While they are out there, the free tiers are not suitable - the closest insist on deleting my database after 90 days. Instead, I ended up separating application hosting, database hosting and sending of email and the below is where I ended up, written in detail in case it helps one of the many others doing this same thing. I did this all back in November, before the Heroku switch-off, and a few months all it seems to all be running well.

My applications

Everything I'm moving is a personal project (no url customisation, limited need for backups, high availability, etc) and a simple 12f(ish) application. They are written in Ruby (mostly Rails, one Sinatra) and make use of Postgres on the backend and some simple mailing. Nothing complicated and Heroku managed all this for me before.

To move from Heroku SendGrid to independent SendGrid, I did find I had to make a small update to the mailing config.

Most of this diff are linting updates - only the domain change should be necessary.

Databases

I chose ElephantSQL as it has a good enough free tier offering (limited to 20mb) and lets the free databases persist. Detailed steps for anyone who, like me, hasn't done much with a Postgres database for a while.

Setup:

• Create account in ElephantSQL (sign in with Github)
• Create database
• Get connection string

Backup from Heroku:

heroku pg:backups capture --app myapp
curl -o latest.dump `heroku pg:backups public-url --app myapp`

Restore to ElephantSQL:

pg_restore --host "machine.db.elephantsql.com" --port "5432" --username "blah" 
    --password --no-owner --dbname "blah" --clean --verbose "latest.dump"

And voila, database ready to rock in ElephantSQL. This can be checked by running up locally, thus:

DATABASE_URL=postgres://blah:blahblah@machine.db.elephantsql.com/blah bundle exec rails s

Mailing

My requirements are very light here since I only use email for "forgotten passwords" email. For this, a free [SendGrid](https://sendgrid.com/) account was more than enough. I'm using SendGrid because this was the Heroku solution, so it makes for an easy switch.

• Create account in SendGrid
• Create API key

Application hosting

I wanted something which, like Heroku, was a container-based PaaS. I certainly can wrap my applications in containers or deploy to virtual machines, but honestly who has the time. I want to point my application at a hosting provider and have it do the work for me.

For this, I looked at a load of options but I settled on two - Koyeb and Render. Both have workable free tier hosting. Render works like Heroku used to - giving a number of hours per month and spinning down applications when they are not in use. Koyeb simply gives you $5 credit per month and lets you choose how to host. This is an enough to have an application running on a low tier package without spinning down, but not enough to run several applications.

For deployment, the steps were basically the same. For Koyeb:

• Create account in Koyeb (sign in with Github)
• Create application
• Add Github integration
• Add to "run" command: rails db:migrate && rails server
• Select repo (this took a while to appear for me, although Github was having issues when I did this)
• Add DATABASE_URL env var
• Add RAILS_MASTER_KEY env var
• Add SendGrid env vars
• And any other env vars

Then off it goes. I did find it took a little while to start working, but it has been fine since. Also, a slight gotcha - Koyeb appears to have some broken validation when it comes to counts, eg "must be 3 characters or more" seems to actually mean "more than three characters".

For testing, I did all this (including the mailing change above) in a branch then cleaned up at the end.

For Render it was much the same, except I didn't need to specify the RAIL_MASTER_KEY env var and I needed to write a build script.

Cleanup

Finally, to finish off:

• Spin down the Heroku app
• Remove auto-deploy to Heroku (in my case from Codeship)
• Toggle Koyeb / Render over to watch "main" if using a test branch
• Make sure the instance is healthy (and make sure not exposed the wrong port anywhere)
• Final redeploy

Roundup

That is pretty much it. I have been running this setup since November with no problems. I am getting fewer notifications through to Slack, so at some point I would like to get better alerting of start / end deployments but since it's just me working on these things I can watch them easily enough - it's not a priority. I'm just pleased I can still deploy easily and free as that means I'll keep my hand in writing bits and pieces.