Streaming media from Windows 10 to Android

I promise I will go back to writing about technical leadership soon, but for the moment I'm rather enjoying solving real problems. I'm revisiting problems I've worked through at different times in the past few years and it's very interesting to see how things have moved forward, opening new avenues to success and generally blurring the lines between professional-level skills to set up services and just clicking around and seeing Things Work.

More importantly, it's just ... fun. And it really helps to remind me that computers are flexible and interesting tools to solve problems, as well as eternal sources of legacy debt and pain.

Anyway, on the less existential end of all this I want to talk about streaming video. Let's go!

The problem

I have lots of video clips on a hard drive. I can watch these easily on my desktop computer, but I'd like to be able to access them on my tablet without plugging it in there.

The history

In the past, I've created a media server on my network. I've attempted to buy something (utter failure) and made one from a raspberry pi (success, although some serious caveats on that). Neither result kept going and I abandoned them for a long while.

The now

While a separate server would be a better solution, for the moment I'm happy just using my desktop as the server. It's usually on, and that's enough for me if I want to go flop on the sofa with a tablet. So - something on the desktop to act as a server, and something on the tablet to receive it.

If you're totally new to this kind of thing, the important standards here are uPnP and DLNA. uPnP is what allows your media server to be "found" on the network. DLNA is built on top of uPnP and is specific to media sharing, ie it adds in the bit that handles the streaming. These are pretty open (in terms of security) so only suitable for home / other trusted networks.

Server first. I'm running Windows 10 (for the moment... sadness...) and after a bit of poking around the internet looking for something to install to act as a server, it turns out that Windows 10 now does this natively! This was a surprise - back in the day I'd have had to install all manner of media server applications and cross my fingers. Now it's a case of:

• Control Panel -> Network and Internet -> Network and Sharing Center
• In the left pane, "Change advanced sharing settings"
• In the Network Discovery section, turn on network discovery and hit Save

Voila.

Ok, the tablet end, which is running Android. I am pretty sure I could do everything using VLC player, but popular opinion online is that I should use BubbleUPnP for discovering the filestore, and it then launches VLC when I hit play. So I did that. And it worked seamlessly. I had to hunt around a bit through the folder structure offered, but otherwise it just works. Done.

I have dodged a significant amount of the complexity here as I am not streaming to a smart TV (because I don't have one). This means that I can use VLC as the player on my tablet, and that is smart enough to handle more or less anything thrown at it - no codec issues for me.

And there we have it. A few clicks one end, and installing an app the other end and we're away - so much easier than before. One particularly interesting thing I found was that searches for "streaming from desktop computer" leads to information about streaming games, not other kinds of media. Amazing how much game streaming has grown - to the point it's the primary search responses.

I promise I've been doing technical leadership and strategic things too. I'll write about that soon...

Email Three - Email with a Vengeance

"You email isn't arriving at all now" - everyone.

I have spent far too long writing about email and how to set up vanity domains. This really should be easy and Just Work but ... well. Here is the third post. Why do I care? Well, given how important email is as part of our online identities I do believe in taking some ownership of it, hence using a vanity domain. By using my own domain instead of an @gmail.com address I could migrate away from Gmail in the future without losing access to everything in the world. While I don't intend to go anywhere any time soon, Google does have a habit of doing odd things with its services so I'd like to have some options (he says, using Blogger which is far more at risk than Gmail...).

With that in mind, I'd like to use a vanity domain. I'd also like my email to arrive. And I'd like people to be able to email me too. High requirements, I know.

The story so far

So this is the third post on this subject (sigh). In my first post I went into detail on my requirements and the underpinning bits of security apparatus required to make email happen. I set things up using SendGrid but lamented using a marketing company for email as well as a cap on my daily email usage.

In my second post I removed SendGrid as sending / receiving wasn't consistent and switched to using the Gmail mailservers. This removed the restrictions but also made it impossible to set up DKIM and DMARC properly. I helped my setup by setting p=none which is better than nothing, but not by a lot.

Guess what? Email didn't send / receive again. This appears to have gotten worse recently, or I'm noticing it more. When three email vanished over a couple of days I cracked - I can't live with inconsistent email. It's too important.

The problem

Reading around suggests that the problem is to do with how email forwarding works. No-frills forwarding essentially throws the email at the receiving server. The receiving server then figures out what to do with it. This is fine, until one factors in load - and that all spam needs forwarding in case of false positives. The system needs to decide what to do when it is overloaded, and it seems the Gmail servers drop email in this case. Then the forwarding service needs to decide what to do and the simplest approach is to also drop the email - else they are then storing email which has its own overheads and problems.

This is a crude explanation - here is an expert explaining it far more accurately.

Considering I've been using free options, I can see why they've taken this approach but it's not good enough for me.

The solution

The solution is to use something which holds incoming email temporarily and retries if the forwarding fails. There are a few ways to do this, including some approaches using scripting and free services but as noted above I'm really bored of fiddling with this ecosystem then gaslighting myself into thinking it's working when there are a few, but notable, errors. No scripts, time for something a bit more thorough.

Enter Gmailify. Apparently Tim O'Neill suggested this to me the first time around, but either I didn't note it or I got confused with the Google feature of exactly the same name. Either way, I am now giving it a go and the pricetag ($7 / year at time of writing) is very reasonable.

Gmailify works as a forwarding / mailbox service. It controls the incoming / outgoing mail on your domain and temporarily lets the email rest in a mailbox. Gmail then uses POP3 to pull from that mailbox which then erases all trace. It also enables all the DKIM / SPF / DMARC setup that was missing before.

Setup is really straightforward if you know how to edit DNS settings and tbh should be easy if you're just confident clicking around. It gives you exactly what you need at each step, and an option to verify each step has gone in properly. The interface for routing different addresses on your domain is really easy to use too, at least for a simple setup.

Couple of things that took me a moment of thought. First, you need to set up the primary email address then configure the catch-all email address if you're used to *@domain.com. This is easy in the Email Routing submenu. Second, Gmail doesn't automatically prompt for outgoing email any more (could be because I was migrating a config?) and when modifying an existing outgoing mail rule it doesn't perform a full validation which will likely create problems down the line. I got around this by deleting my existing outgoing mail rule and setting up from scratch again. Don't forget to reset your default outgoing email address if you do this!

Oh, and if you're migrating rather than setting this up for the first time don't forget to clean up your DNS config when you're done.

All done in less time than it took me to type this up. I sent some email to Tim's overly-fussy email account and it all got through which is a first. I also ran it through this awesome tool for learning and testing DMARC settings which is worth a play if only to see how education tools should be designed. All the tests now light up a pleasing green - another first.

I've had this set up a few days so I'm keeping my fingers crossed this is the last time I have to write about this...

Looking back and remembering

I intended to start this year with a post about what might be coming in 2025, filled with enthusiasm and excitement. Instead, with the passing of a friend, I want to write a few words about remembering the important people in our working lives and what makes them important.

Over my career I've met a lot of people - some have been great at what they do. The people I remember are the ones who are more than competent, they are decent, authentic and overall nice individuals. "Nice" can sometimes be a little derogatory - a substitute for anything more positive to say. But, for me, the people who properly exemplify "nice" show off the traits we claim we value the highest. They encourage and provide positivity. They create non-judgemental space, while offering constructive feedback. They support without coddling. They make time for people and help build up those around them, usually managing to do so without making themselves the centre of attention. They usually do this somewhat unconsciously - it is their natural way to help, build and encourage. Nice people have a positive impact on their world.

Truly nice people are rare, but they are out there and should be valued. Even more so, we should value the people who remember those who have passed out their immediate sphere (say, changed job or moved away) and keep those connections going. A friend of mine refers to these folk as "glue people" - people who bring others together, often with as simple a thought process as "you're a decent person who has a problem, I remember person X who could probably solve it - you should talk". In this world of networking and personal brand, these glue people are surprisingly rare and those who do it authentically and honestly (ie not attempting to become some kind of power broker) are even rarer. They help hold extended groups together and create new connections through truthful endorsement and recommendations and lots of people owe personal and professional change and development to them - often without even realising it. We are collectively poorer when we lose them.

I don't like naming individuals on this blog, especially when I know they can't ever respond or correct or ask me to take it down, so I'm not going to do this today but I know we have lost someone decent, authentic and nice and I know they have made a huge difference to my life, and the lives of others. We may not fully recognise it. Frankly, this person probably didn't realise it either and that is both wonderful and intensely sad.

Life is short and frail. I'm starting this year thinking about who I want to be, and what I'd like people to remember me for.

The year that was, 2024

The end of the year approaches and another strange one. Far too much work, and I really don't feel like I've done a lot. However, the actual results are quite different so I'm not sure why I feel less than satisfied with myself. Let's dig into the detail - there is certainly more than Helldivers 2 and Dave the Diver.

• 13 posts on this blog (14 including this one) - tenth consecutive year of a post per month
• Continued posting my blog posts on LinkedIn and getting some good responses
• Passed my Magistrates application interview
• Another year of the Year in Pictures - tenth year and we're up to 47 photographers
• 347 Github contributions - well up on the last few years
• Including extensive rewrite of styling for Year in Pics site, removing old Foundation 5
• And a performance rewrite which cut build time from 85 seconds to 6
• Complete upgrade to all Ruby apps and code including migrating Postgres databases (again)
• Changed my email setup again - sigh
• Resurrected an old Pixel C
• Rebuilt my desktop computer
• Updated my music system
• Another year of Saturday Pain and made a t-shirt
• Learned to make candles
• More cooking
• Started writing D&D again
• Started bread again! Although not a great first run...
• Made a Lego Piranha plant, Optimus Prime, and URB Studio
• Made a dice display
• Showed my photos at the St Swithins creativity event
• Pie making cookery course
• Ran quarterly dinner parties - a fancy name for games and food
• Made a few different photo decorated items and books
• Escaped another couple of escape rooms
• Another year of exercise with a personal trainer
• Ongoing good physiotherapy regime
• Spoke at a conference about AI in the charity sector
• Another year on the Data for London Advisory Board
• Yet more instagramming
• More photos on my Flickr stream

Resolution count - 5/10. Not bad, and a lot better than last year.

So yes, this has been an odd year. I've achieved a lot more than I thought despite a rough year at work. A year ago I wrote this, for the millionth year in a row:

Looking forward to 2024, I'm going to write the same thing I write every year. I need to spend less time working and more time living.

Well, I'm happy to say I worked it out this year - I finally found a way to regenerate my energy and spend more time on the things I enjoy. Unfortunately, that way was leaving my job so it feels a bit all or nothing...

Overall though, I have two thoughts about 2024. First, I am quite pleased with how much I did this year. This time last year I struggled to write the above list, and really felt I was bulking it out with non-events. This year I managed to write a lot more and the general quality feels a lot higher. Good news! On the slight downside, far too many of these are from the last couple of months of the year - that is, after removing work. So I still have no idea how to balance my life properly - hurrah for adulting.

Second, as I said in the intro I don't feel like I've done a lot. The evidence is there - I'm wrong and I did Many Things - so why is the emotional component not? Some is because the year has been backloaded, so for much of the year maybe that feeling is more accurate ... but still. What would I feel to be a successful year?

My low energy levels is likely partially a cause. For the second year running I have ended the year with another series of blood tests for fatigue - clear for the second time, fortunately, but the fact I keep having these tests is not a good sign. I really need to bring my physical and mental health to the point where normal fluctuations don't keep dipping me into a bad place.

But overall, I don't have a good answer to the question about why I am not satisfied with my own achievements. I do have a bit of an opportunity at the start of 2025. Being between jobs I'm at a natural point to be introspective and really think about what I want to achieve going forward, and also what would make me happy. So 2025 is going to be the year of working out where I want my life to go, along with actually doing some of it. Oh and getting a job, I guess. Boo.

One thing I do need carefully consider is how I interact with other people and the world. I have definitely withdrawn over 2024, mostly as a defence mechanism. Through tiredness and some bad experiences I deliberately focused on the things in my life over which I have some control, which is to say I looked inward. This helped, but I think the balance is off. I have the inclination to become a hermit if left to my own devices so I need to make sure I am remembering to look outward too.

Right, enough ruminating. 2024 was objectively a decent year, if hard work. I hope for more from 2025, but to add in the emotional component. In the 2021 version of this post, I wrote that I wanted to find and restore my sense of wonder. I think I did, but it has slipped again. So this year I want to bring it back again.

It is time. Onwards! To 2025!

Leaving Macmillan

Back in mid-October I closed my Macmillan laptop (my MacTop, if you will) for the final time, leaving my role as Director of Engineering. I left behind the Engineering division I had created and some excellent and passionate people - not an easy decision and I'm still working through how I feel about everything, including the wider Tech industry. This is one reason it has taken so long to write this reflection.

I was at Macmillan for nearly three years. When I started, the organisation had some in-house technical capability but the people were scattered around working under project managers on systems that were badly in need of modernisation. I can write lots about how we created technical strategy, developed a software engineering culture and community and brought in greater automation, and all of this would be true, but the most important work was bringing the software engineers together and giving them a voice. The most important work is always with the people.

Over these three years I've seen many people grow into new versions of themselves by giving them space and encouragement. Some have embraced the wider context of their roles - looking beyond the specific technical problem to why it's important and realising they have a voice in this space, which can result in better solutions. Some have taken better ownership of their work, owning processes rather than simply implementing them and then flourishing as they realise they can improve and refine. Others have been able to express the needs of a previously ignored service - very common with infrastructure elements - and deal with issues and annoyances to move those areas forward. Still others have taken on line management or team leadership duties and started their journey into wider leadership activities.

I've spent the last eight years or so of my career trying to show the value of technical voices at all levels in an organisation, so having the opportunity to demonstrate this in action while enabling lots of people to grow and flourish has been great.

On a personal level, I've also grown a lot in these three years. I've learned a ton about being part of a director cohort in an organisation, and this is the first time I've worked in the charity sector. I was also recognised at an industry event as part of a group of up and coming "next CIO"s which is not only fantastic for the ego, it's also something I wouldn't have even considered a couple of years earlier. I've experienced several forms of restructure, all of which have taught me something new, and I've been responsible for shifting organisation culture in ways that go far beyond what I've done in the past.

I've also worked with some amazing people who have helped shaped my thinking and career in hugely significant ways. I'm not going to name names, but if you think it's you you're probably right. If I worked closely with you and you DON'T think it's you, you're probably wrong. In both cases, thank you.

So why leave? Primarily, the organisation needs to crack some really knotty legacy technology issues. I've been working on these since day one, and I believe I've built foundations on which solutions can now surface. However this has been a long journey, including migrating physical server rooms and a load of hiring and restructuring, and I did not feel I was the right person to take things forward in the next phase. The org needs someone fresh and enthusiastic for the challenge and after working on this for three years, I need a new problem to re-motivate me.

However, I am still sad to leave behind some excellent people - some of whom I've helped make serious changes to their careers. I'm going to miss them, and I hope I can work with them again. I've been part of a huge change in Macmillan technology, and this is something for which I can feel some pride. A friend says that a mark of success is leaving the place better than you found it, and I think by that metric I've done ok.

Looking forward, I'm enjoying some time away from work and considering my options. Reading back my Leaving GDS post was an interesting insight - at the time, I said "I'll likely jump into another maelstrom of "interesting problems"" and this proved to be rather prophetic. I am my own worst enemy when it comes to containing work and maintaining balance with the rest of life so I'm going to embrace this break as another opportunity to reset myself. Certainly, I'm currently enjoying the freedom to work on my health and fitness and learn some new skills. I've been making candles and writing lots of code, as well as starting to write again - all things I've been too tired to do while in role. I really want to find the not-work person underneath and bring them out enough to do a better job surviving the next job.

I'm considering my next professional steps carefully, especially how much Data I want to be taking on going forward. Do I want to push in the direction of a Chief Data Officer role? Do I want to stay very much in the core technology space? Regardless the answer, this is certainly something to look at in any job spec to avoid adopting data through the back door into a software engineering role, which I feel will set up a role for disaster. The tech industry is going through another seismic shift - we are in the early stages of the AI revolution, with everyone racing to "do data and AI". There is a huge software engineering component to this, but it goes a lot wider into organisational process and culture, and this work needs clear air and appropriate reach to have any chance of success without being overloaded. Fortunately, culture and technology is an intersection I have a lot of experience working on so there is certainly also potential here.

Before jumping anywhere, I need to revisit my own motivators and see what I actually want to be doing. Macmillan has been amazing experience, but I want to build on this, not just do it again somewhere else without some serious thought first. Last time I wrote one of these posts I said I was keen to continue promoting technical leadership and I feel the same way now, but I need to decide how.

In the meantime, if anyone would like to talk about the tech industry, data, growing your software engineering capability, people or anything else please do give me a shout in the new year - I'm easy to contact on LinkedIn or Twitter.

Right now, I'm enjoying the peace in that strange time between Christmas and New Year when the world is on pause and nobody expects anything of you.

Resurrecting a Pixel C

I'm putting off the post I need to write so I'm going to ramble a bit about resurrecting an old Pixel C tablet by sideloading a custom operating system. This is something that sounds scary but really isn't, so I thought I'd share in case anyone else has hardware in the cupboard and no desire to buy new, expensive kit.

Setting the scene

My tablet usage is pretty modest. For anything significant, I'll use a laptop / desktop or my mobile phone. My tablet is mostly used for video streaming (YouTube, Netflix, etc) and a bit of web browsing when it's the closest thing to hand. I am not a power user.

Many years ago (2016), I bought a Google Pixel C tablet. This was originally released in 2015 and was Google's first tablet under the Pixel brand. Other than having a name that is really annoying to search for, it was a fabulous piece of kit - feeling very solid and chunky. However for boring reasons, when COVID hit I moved and my Pixel C fell out of use.

Fast forward many years, and my iPad battery is dying so I thought I'd see about resurrecting the Pixel. Firstly, I hadn't realised how long it had been since it was last used - like a lot of people I know, my sense of time has been utterly smashed by two years of lockdown - so it was a bit of a surprise to discover Google stopped supporting it nearly six years ago (end of 2018). Consequently, although it fired up fine, it was hideously out of date with no path to catching up. It seems older versions of Android (8 in this case) have a problem connecting to modern 5ghz wifi connections and so my lovely hardware was dead unless I wanted to run a lower speed wifi network.

Why take the easy solution, eh?

So, with my brain telling me "this is only a few years old" (it's nine years old?!) I thought I'd look at bringing it back to life via the medium of a custom ROM. For the uninitiated, this is basically installing a new operating system package - pre-built with proper drives and so on - however is more complicated because tablets and phones tend to lock all this down to stop people fiddling and bricking the device. However, I'm (apparently) a grown adult so I'll fiddle if I damn well please. Plus, it doesn't work now so it's not going to get any less functional - perfect for some learning.

Get on with it

The hard part was really all the prep. I needed to find a good ROM, which I did via the application of Google and reading. LineageOS is a the gold standard for community-run operating systems, but even they have dropped support for the Pixel C. However there are some intrepid folk out there who are keeping the dream alive on the XDA forums and a helpful dev going by npjohnson is pushing out builds for the Sphynx project, via his forum thread. Sphynx is an unofficial build of LineageOS tailored for the Pixel C - perfect for me.

The instructions are good - I read them a few times before giving this a go because I was scared - and basically there are four stages:

1. Download adb and fastboot to your computer - this laptop is Ubuntu Linux so that was as simple as sudo apt install adb fastboot but Windows options are also available
2. Learn how to boot your tablet into the recover mode menu (for the Pixel C, with the device off hold Power and Volume Down)
3. Download the desired image - I just took the latest (lineage-21.0-20241019-UNOFFICIAL-sphynx.zip at the time) and it worked fine
4. Follow the instructions very closely

Extremely minor gotcha - I extracted the downloaded zip file to get a recovery.img file for page 3, then used the original zip for sideloading in step 4. Other than a couple of slightly alarming beeping sounds from the device, this was the only time I really needed to think once I got going. The whole process took about an hour, going very slowly and carefully, then additional time to set up the tablet (obviously it is wiped so you'll lose anything on it from before).

Behold

And it's working! There are some known issues - the camera doesn't work properly, apparently bluetooth is a mixed bag and the rotational sensors also don't work - but these haven't impacted my needs. If you like running your tablet in portrait mode, apparently this can be a pain. However, for me I have a shiny refurbed tablet that plays videos and doesn't keep turning off mid-video. Hurrah!

Overall, this is scary. But it turns out it is also easy. Hopefully others will give it a go and bring their devices back to life.

Sending email - redux

It feels like forever since I wrote my last blog post on sending email via a vanity domain but has actually only been a year. In that post, I noted that sending via SendGrid was optional and it should all be doable using Google servers. The SendGrid config has been mostly ok, but hasn't been perfect and I wanted to remove this free third party from my email tool chain so I've revisited the setup and got it working through the Google mail server.

Will this work long term? Hopefully, but I'm not convinced for reasons I've laid out below. Let's do this.

Sending email

First I needed an app password for your Google account, which is a bit of a buried in the security interfaces. You can find the admin console for app passwords here.

This is also reachable by going to Google account settings, under "How you sign in to Google" select "2-Step Verification" and the option for App Passwords is at the bottom of the page.

Next, to configure gmail to send via the Google mail server, I needed to set the outgoing mail, found in:

Settings -> See all settings -> Accounts and import -> Send mail as

Then adding / editing my intended outgoing email address with these details on the second page ("Send emails through your SMTP server"):

SMTP server: smtp.gmail.com
Port: 465
Username: Your google account (blah@gmail.com)
Password: Your app password from earlier
Secured connection using SSL ticked

Email security

This is a minefield and something I'm going to have to monitor to see how horribly I have broken things. First, this tool from Red Sift is great for analysing email security settings.

Deep breath...

SPF

To the DNS record, add:

include:_spf.google.com

and remove references to sendgrid to avoid too many lookups - this flags it as insecure.

DKIM

Without a full Google Workplace account, I can't upload a private key to Gmail so DKIM isn't going to work. Hopefully SPF will be enough. We'll see...

I think this also scuppers MTA-STS, but happy to be corrected.

DMARC

This is tricky. DMARC requires one of DKIM or SPF to pass tests properly in order to pass, then it suggest the receiving server takes a specified action (via the p flag). In this case, my DKIM check is going to fail so that is out. The SPF check passes the initial test however there is a further check to make sure the Sender and From headers are the same. In my case they are not, since Sender is gmail and From is my domain so check fails with a no-alignment error - thus the DMARC check itself fails.

I've "solved" this by setting the p flag in my DMARC DNS entry to "none" which just tells the receiving email server to deliver it anyway. It seems to work for my very limited testing sample, but obviously I'm not happy about this approach.

What is next?

I hope the SPF config will be enough to make my email work happily again, however I'm clearly hitting the limits of the free options in Gmail. If this doesn't work well enough, I think I'll move away from free options and look at something like AmazonSES which (from a quick read) will let me configure everything I need and charge me $0.10 per 1000 outgoing emails. This is probably the ultimate "correct" solution (unless I want to pay for a Google Workspace account) but is a lot more work and right now I don't wanna.