Sending email in 2023

"Your email keeps going into my junk box" - everyone.

I use a vanity domain to front my email address. I used to run a simple setup where the domain was basically masking my Gmail account. Incoming was handled by a wildcard forward in the domain host. Outgoing, I simply rewrote the email envelope with my desired email address. Essentially I was spoofing the outgoing email.

Gmail used to let me do this, but clamped down years ago requiring proper authentication with an SMTP host however the old setup still worked, as long as I didn't change anything.

Then the big email providers started clamping down on this kind of thing. In an effort to combat spam, email is increasingly complicated and the wider ecosystem is getting more locked down. There is a big rumble about the big providers essentially pushing smaller email providers out by blanket not trusting them, making it increasingly difficult to run your own email setup. This post is not about that, rather it's how I stopped my email started going into junk boxes. I was forging my own sender address, which is exactly the kind of behaviour you see from various types of spam. Nice.

So, on the assumption I wanted my email to arrive I needed to revisit my configuration and set this up properly. I did a bit of work, so I thought I'd write up here so I can repair it in future if needs be, and it's in one place on the offchance it helps anyone else.

Incoming email - you're emailing me

Not many changes here - although I use a combination of Cloudflare and Ionos DNS these days, but a blanket forwarding rule in the Ionos config from the whole domain still works.

Outgoing email - I'm emailing you

Ok, this is where it gets interesting. I can still send email, setting the domain to whatever I want, but my emails are being flagged as spam. This is because the receiving hosts are trying to protect the account owners from spam and my setup was being flagged as spam. Obvious note - I set up a test Gmail account for receiving email so I could test the effects of my settings.

Outgoing SMTP server

First thing was properly configuring an outgoing mail server. In theory, this can be done with the Gmail SMTP service but while I could authenticate properly I found my email still ended up flagged as spam. I'm sure there is a way to do this properly but for the moment I instead turned to SendGrid and this documentation was useful.

A free account allows 100 emails per day - plenty for me. Nobody wants to hear more of me than that. In the SendGrid interface it is easy to create a API key (Settings -> API keys) with appropriate emailing sending permission then when adding the server details, just select smtp.sendgrid.net / apikey / $YourKey. Only slight gotcha is making sure you get the port right (SSL over port 465). This should authenticate properly and email can be sent - although it'll probably be going to junk again.

Next up, setting up DKIM. This stands for DomainKeys Identified Mail - an email authentication method designed which allows the recipient to check that an email came from the domain it claims, and was allowed by the domain owner. The setup is found in Settings -> Sender Authentication. You might be able to get away with Single Sender Verification, but I did the full Domain Authentication. You need to be able to modify your domain's DNS settings for this to work properly.

If the setup doesn't seem to be working properly you can test the individual additions on the command line with a tool like dig.

dig foo8908.tomnatt.com should give a NOERROR response. If it's not, the setting isn't right or it hasn't refreshed yet.

Finally, assuming this is for personal email you'll want to disable link tracking. This rewrites links in your email for marketing purposes and likely break any links you send unless you configure it properly. Turn it off with Settings -> Tracking -> Click tracking -> disable and links will work again.

Other DNS setup

There are two other DNS entries that can help with proving email provenance - SPF and DMARC. I'm not sure whether I needed all these for a minimal setup, but they do work best when all three are present. I did configure them, so I'm capturing what I did. 

SPF (Sender Policy Framework) is another way to ensure the mail server sending an email is allowed to send via this domain. It works by defining which servers can send email, so the client can check, rather than directly encrypting the connection (the DKIM approach). The setup is fairly simple, and can be checked with tools like this.

An SPF policy which allows sending from Gmail and SendGrid servers might look like this:

v=spf1 include:sendgrid.net include:gmail.com ~all

DMARC (Domain-based Message Authentication, Reporting & Conformance) helps receiving mail systems decide what to do with incoming mail that fails validation via SPF or DKIM. So this is worthless without at least one of the other two.

A rule which tells the receiver to mark failing email as spam and send reports to the given email address would look like this:

v=DMARC1;p=quarantine;pct=100;rua=mailto:postmaster@tomnatt.com

Done

And lo, email appears to be flowing again. I hope something here helps. To finish, I want to note that I'm not an email expert - not even close. If you are, and you're seeing somewhere I've written something stupid please reach out and I'll correct and attribute.

A few words for World Mental Health Day

Earlier in the week (Tuesday 10th October) was World Mental Health Day. As part of normalising mental health through openly talking about it, at work I posted something of my own experiences on our internal chat board. The reception was extremely positive - people pleased someone spoke out, feeling seen and so on. So - since mental health isn't just about last Tuesday - I'm reproducing a version of that below, albeit with the work-specifics removed.

For World Mental Health Day I wanted to share something personal. Mental health still has a social stigma attached, and while this is improving rapidly even from five years ago there is a long way to go. So, to make my small contribution, I'm going to tell you a little about myself.

I have suffered from depression throughout my adult life. I'm fortunate enough that this has very rarely reached the more intense levels where I can't get out of bed or worse, but I've certainly had ongoing difficulties with anxiety and feelings of emptiness, loss of interest in life in general and feelings of hopelessness, guilt, worthlessness, helplessness and others. Even writing this makes me feel like a fraud, since it's not "proper" depression etc etc...

I know that these feelings go in cycles and I've spent a lot of time identifying my own triggers so I know when I need to take a step back and protect myself before I get into a worse state. I also know that it sometimes sneaks up, and I suddenly find myself in a bad way. It's hard and deeply unpleasant at times.

I'm sharing this mostly because I don't think people talk about mental health openly enough, and World Mental Health Day seems like a good day to put something out there. For some people, I hope to show some solidarity. For others, this will be some small insight into a world that is totally alien to you.

On this blog, sometimes I post about mental health. If you'd like to see what I wrote after taking about a month off you can see that here, and you can see all my posts about mental health here. Obvious disclaimer - I am not a medical professional.

Most important - if you're in a similar position, know you're not alone. There are people who share your experiences and are willing to talk. If anyone would like to talk to me, please feel free to reach out. I'm also quite happy for the "but what's it like" conversation if you'd like to ask that.

At work, I suggested a discussion group and it occurs to me that that might be a useful thought in other contexts - either for people who have different experiences they want to share, or for a wider group to talk. If people think that is a good idea do shout and if enough people are interested I'll see if I can get something set up.

Take care everyone.

Some thoughts about the future of the Tech industry

Last week I was given the opportunity to sit on a panel of technical leaders and talk about the future of technology. I had a few notes about about how we're going to need to change our thinking about building capability and I thought I may as well capture and flesh out a touch the results of my crystal ball gazing here.

I spoke briefly about three areas:

• The people we hire
• The expectations of our users, and our expectations of them
• Where I think we’re going to need to invest and build capability

The people

Fairly obviously, Technology is getting more important to daily operations. But it's getting harder to hire people all the time. As we all keep hunting for talent, those who aren't offering the top end salaries will increasingly have to look nationally or even globally to recruit. I don't think moving the organisation to another city is a sustainable approach - at best, it will simply move the problem. Instead, I think we will increasingly see a more distributed workforce, and therefore more remote working. As staff turnover is identified as a major organisational cost, we’ll also see more emphasis staff retention - succession, training, individual growth and so on.

Whoever nails building a strong remote working culture and environment which encourages loyalty and celebrates and develops the individual is going to do very well. I think the secret to this is going to be building very strong communities of practice, and if I’m right we’ll see more “Head of Community” style work and roles growing up. 

I also think we’ll see more Tech decisions based not on the best technical or product solution for the org, but the best fit for the skills we can grow or hire. This would suggest a lean towards the big names (the Microsofts, Googles and Amazons) who are heavily investing in training the Tech industry through free access to courses, sponsoring hackathons, and so on.

I believe this will be more acute as Automation is used to deliver more while avoiding the continual growth of IT departments. We will need to retain the skillsets to maintain that automation layer or we'll be seeing yet another wave of technical debt.

Expectations from and on users changing

In times of yore, users used to have to know something about operating a computer to install and use software. These days, users on a smart device can just touch an icon and get everything they need. This is a victory for accessibility and digital inclusion but it also means the gap between “technical” and “non-technical” user is widening. Our helpdesks and other support points will need to work with an increasingly broad ranges of questions, especially as tech like AI gains traction, and expectations for what it can do are all over the place. 

We’ve also been seeing for years the expectations from users increasing as they use more SaaS products at home and demand the same sort of tools at work. To satisfy these needs, the cost of development is going to go up and cover a wider range of skillsets - and of course this links back to the earlier points about skill availability. Out of the box services are also going to be affected - vanilla deployment is going to be less palatable in the office, requiring more work for a good result especially with the current state of many internal systems user interfaces with respect to accessibility and usability. 

This is particularly true regarding what have often been considered secondary requirements - accessibility and environmental sustainability for example. Users are (quite rightly!) far more vocal about accessibility needs, and we need to not just respond but get ahead of their requirements.

Other places we’re going to need to build capability

Technology is obviously an increasingly essential part of everything. I mentioned the effect on helpdesks above. We're also already seeing increasing amounts of security threats and the wider reaching impact of a successful attack. This will take us into an ever more expensive arms race in the Security field, which will mean building Security capability. This is going to need to be approached very carefully as it will be very expensive - everything I’ve said about skills shortages are far more acute in the world of InfoSec. Part of the Security picture is a renewed emphasis on good, basic engineering practice (such as patching) - but again, this places a challenge on building skills in our organisations.

We’re also generating and handling more data all the time, so inevitably we’ll see more human error leading to data loss. In fact, for any organisation a major security incident or data breach is only a matter of time now. If we are assuming that it is going to happen, there is a need for much more robust organisational responses to these scenarios which means building appropriate incident response and Business Continuity capabilities. Of course, just responding isn't enough so there will also need to be stronger data ownership throughout our organisations, with more people with data owner and controller roles. Organisations will need to fully grip their end to end processes and user journeys in ways that perhaps hasn’t been happening before.

Obviously there is a lot more that can be said about everything here!

Looking to the future

One of the symptoms of seniority that I still struggle to process is this idea that people want to hear what I have to say and understand how they can (professionally at least) be me. It's a weird feeling, both humbling and frightening, and has only grown in the last few months. I'm being asked to speak at conferences, and in a few weeks I'm on a panel discussing the future of the Tech industry. Why? Surely there is someone better for this?

Honestly, this is a question I've been asking myself since my time in the Civil Service, where I found myself in a meeting discussing how to spend five million pounds of public money. I should write a post about getting over myself and getting on with it. This, however, is not that post.

I moved away from development into management so I could make a difference to the industry. How do people get into Tech? How do people move to more senior technical roles? Showing the importance of senior leadership with technical backgrounds, and helping people get the right experience to be credible candidates. These are core to my motivation and it seems I have the beginning of a platform. This is a responsibility I want to take seriously. What can I offer to the wider industry?

I've got a few ideas and since there seems to be a few people reading this blog from LinkedIn, I suppose I can offer them up. First and foremost, I'm thinking of creating a set of interviews with people in my position telling their story, talking about how they got to where they are today. I don't kid myself that I'm some kind of Steve Jobs or Bill Gates, but then neither are the overwhelming majority of people. I am, however, fairly successful in the industry. I'm in a position that someone getting into Tech can reasonably aspire to attain. Not inspirational perhaps, but practical. There are lots of people like me, and the stories don't get told often enough so I like the idea of creating some kind of resource showing the different paths people can take. At the moment I'm thinking a podcast, but format to be decided. If you're reading this and think it's an interesting or useful idea, please do reach out.

In addition, I sit on the London Data Board, and I'm happy to offer some time to support other initiatives if they come up. Again, do reach out - especially if they are something to do with nature or animals. On top of this, I already do some mentoring, and I've got a whole post about that coming up soon.

I opened saying that people asking about my career and how to "be me" is weird. Actually, on reflection, it's a weirder feeling that I'm in a position to be feeling a sense of responsibility to the industry and asking any of the above questions. It's a great privilege to be working somewhere where I can spend some time looking outwards and hopefully make a difference. That's my intention anyway - so I've written it down in public so I can hold myself to account.

A break in the routine

When I started as Director of Engineering at Macmillan, one of my objectives was to build a department which functioned without me. I wanted to be able to pull myself out of the operational day to day, empowering others to own those problems, while I focused on overall direction and wider concerns. This was always an aim rather than a hard goal - similar to the sysadmin end game of a fully automated system, it's something I would like to chase and I think I can get close, but I don't believe I will ever truly achieve.

Over the last year and a bit I've set things up to work this way. I've created core areas of my division, and put someone in charge of each of those spaces. I've empowered them to make decisions and own their successes, and I've carefully worked with each of these leaders to identify and remove weaknesses. There is still a long way to go, but I think we've made a strong start.

Why write about this? Well, I took last week off as leave (booked some time ago) and at the end of the week:

The Boss: How are you doing?

Me: Starting to feel better! After about four days asleep, I finally started to feel human again.

The Boss: Want another week off?

Me: Is this some kind of trap?

It wasn't a trap, so after a quick conversation I decided against looking that gift horse in the mouth and spent half an hour with my assistant going through my diary. The first week off was all planned, so that hadn't been a problem but this second week was a spur-of-the-moment thing. Surely it was going to cause all kinds of problems if I wasn't around?

Nope.

Of the week, there was an hour which absolutely required me and needed rescheduling. Which means ... what? This is the question I've been pondering for a few days. On the one hand, my ground-in desire to be needed and adored took a massive hit. Being basically superfluous is not good for the ego. On the other hand, this is exactly what I've been building towards. The department can survive without me. It can make decisions, move forward, solve problems, etc etc...

We've got a way to go. The wheels most certainly will not stay on the wagon indefinitely and the more forward-planning functions haven't been tested in this short term. Plus, of course, people are covering  some bits of my role and I shouldn't underplay their help. There is still plenty of need for someone in my position to keep things steady and move things forward - I don't think they can make me redundant just yet. However, I'm pretty pleased with the results of this (accidental) test and what it means for changing my day to day when I'm back.

To be clear - I'm not wasting my time. The bits that didn't need rescheduling were parts of the job where I'm providing support or oversight, and direct support for individuals. All this should be done, but I can certainly ask some questions about what happens if I dial it back a bit. That means less oversight, but perhaps we're ready for that? Then I can focus more on the work I'd be doing myself which isn't getting done. However, the nature of the job is such that my work is (supposed to be) much more strategic which naturally has a longer burn. If it's delayed a week, that doesn't create the same kind of problem as operational work stopping. Fortunately, this also means focusing in on the part of the job I particularly enjoy.

So, in summary I am very deliberately in the process of making myself operationally irrelevant and it's time to reboot my focus list. Is this ... is this what success looks like?

Next CIO 2023

Well, this month I won an award.

Honestly I'm not sure how to write this post. I do not usually deal with praise well, and that is unlikely to change. However an industry award is something to celebrate so I'm going write something about being recognised, and I'm going to keep self-deprecating jokes to a minimum.

Next CIO is an industry award where a group of expect-to-be-CIOs-soon are recognised as the future of the industry and celebrated. They are brought together over the year for workshops and group learning, and hooked up with a mentor - someone who is already a successful CIO.

And I am one of these people. Which means I'm part of the future of our industry. Good grief.

What does this mean? Well, over the next year I'm looking forward to talking with and being mentored by the very generous Wayne Clements, and I'm going to be attending a series of workshops with my Next CIO cohort. I've also got access to a group of excellent people from across the world of Technology - there has to be something useful I can do with that set of contacts.

For the award event, we were asked for volunteers to talk for a couple of minutes about ourselves. I volunteered mostly because while I didn't want to do it, I wanted to make sure I was making the most of the opportunities. That is what this year is going to be about for me - taking opportunities and seeing what happens. In this case, what happened was a talk which got a few laughs (in the right places!) and some excellent conversations in the bar afterwards with people who were interested in what I had said. Only good things.

Looking forward, I'm going to do a few things. I'm going to start posting my blog posts on LinkedIn. Long-term readers will know I write this mostly as a tool for self-reflection. Putting them up on Twitter and Facebook was a big step and LinkedIn will be another (slightly terrifying) moment of development. So ... hello if you're reading from LinkedIn. There are lots of other posts here - you might be interested in the essay subset.

Coming back to the topic of my "about me" talk - I need to think again about how I give back. My particular interests are around how people get into the Tech industry and how they move around and up. Over the years I've coached and mentored, and I want to lean back into this. However, this doesn't scale terribly well - there are only so many hours in the day so there is only so much time I can give - so I want to look at other options. There are a lot of stories out there to be told, and while some people have inspirational tales of mega-success, I think there is a lot of value in the more attainable stories of people who do well without being the next Jeff Bezos or whoever. Can we get better access to those tales?

Anyway, these are thoughts for the future. For the immediate now, I'll end with some pictures from the event and me talking about how I should have been a forester.

Making a decision

I've been thinking a lot about governance recently. In general, I like consensus - it's important that expert voices are heard and action is taken on the back of expertise. I dislike dictating how things are done for the same reason. Generally, I find people thrive when given the space to take ownership of problems and work through them.

However. This can create a few problems. The first is mediocrity - if everyone is compromising to form consensus then it is far too easy for nothing to actually be good. In the worst case, the important part of a proposal is compromised away and any outcome becomes a waste of time. This assumes there IS an outcome, which leads to the second problem - how is a decision actually reached? How do I stop people talking forever?

This latter problem is the one that concerns me more at the moment. I see different versions of non-decisions all over the place, from email threads which ambiguously leave "someone" to do the next thing to conversations which always need another person to engage before maybe everyone agrees. At worst, this never concludes but even at best it is sloooooow.

For me, this leads directly to the question "what is a decision?". When is something approved? When is it agreed? Or when it is simply some positive noises coming out of a discussion? It's important to draw this out for everyone involved. Individuals do not like being misquoted ("no, I didn't actually agree to this"), project managers deserve to know where they stand with sign-off so they can move on ("is that a decision?"), and I need to add some formality so I don't have to mediate these positions.

I'm leaning into some hypotheticals here - while I see some of the above at work, decisions are certainly being made, and people are thriving in their empowerment. However, and being selfish for a moment, it is actually my needs that are being neglected. As my role changes (and reflecting on my musings from my time covering as CIO) I find myself with less and less time to mediate circular conversations. So while things generally work, I need to move this from "generally" to "always" and "with less effort from me".

It's time to make some changes. As noted above, I've previously been happy to give people space to feel things out and learn from them. I don't want to lose that - it's an important part of empowerment and learning. However that is the discussion part - I need to add some stronger gateway decisions to the way we work. On reflection, I think I need to be better at separating discussion from decision and creating a formal record. There is too much going on at the moment to be imprecise about the state of decisions, so I need more opportunity to eyeball people and tell them to speak up or forever hold their peace.

The trick through all this will be retaining agile working patterns - I don't want to create some kind of decision auditing circus, but as we mature we both want to and need to be able to hold each other to account properly. This means having something to be held to - robust decision records where people can make their decisions known formally.

Initial muse for the moment - I've been exhausted this month. More on this later I feel.