Screaming about server updates and rebooting Windows

Maintenance – It’s a word you don’t have to hate anymore.

by Dave Brierley-Jones

Maintenance.. a word synonymous with cold sweats and shivers.

Whether it’s in the context of taking the family car in for ‘Maintenance’ or perhaps ringing a plumber to come and do some ‘Maintenance‘ in the bathroom. The word ‘Maintenance’ when used in a sentence is not often too far away from the word ‘Pain’ and in Radio, it’s no different.

In todays changing world, it’s crucial that Microsoft Windows security updates are applied to our servers and computers to keep things secure. These updates now appear to be rolling out more frequently than ever, and often require a restart of the computer they are being applied to. This gives a challenge when the computer we are updating is the server of our automation system.

In the past when I’ve worked at radio stations, a second server is often pressed into action when a restart needs to take place on the main server. This however usually requires the running of numerous manual tasks to bring the secondary server in sync with the main server. All client machines are then asked to manually flick over to server 2. No matter how many times you do it, it’s never a simple or easy task and something generally doesn’t go to plan.

There is also a big problem with flicking over to a second server, as it’s often running a backup image of server 1. This means that when you make a change to the data on server 2, it’s going to be lost when you flick back to server 1…. so why even bother? Much easier to just wait for server 1 to restart, go make a coffee, have a cigarette and watch an episode of Breaking Bad (maybe two episodes if it’s the 1809 Windows Creator update), returning when the blue ‘Welcome to Windows, we’re just preparing things for you’ message appears on the main server screen. This is all great for production machines but when it comes to the On Air Studio ‘Houston, we have a problem’.

So do we:

a) Mixdown an hour (or two if it’s the 1809 Windows Creator Update) of content from the log and play it from a USB stick or CD?
b) Play a CD and link between each track, reacquainting ourselves with how it used to be?
c) Move from server 1 to server 2 and continue as normal, understanding any changes we make will be lost when we flick back to server 1 after the reboot?

Let’s go for option C… but wait… how do you flick from one server to another whilst still maintaining output? The answer of course involves Option B. As one engineer mumbled to me as he came into the studio ‘Play a track kidda from the CD whilst we switch you over’. These days the cool kids will use Spotify or even YouTube but still, all this fuss and faff is not ideal, especially when these updates may start to become a weekly task. Presenters are also creatures of habit. They don’t like change or to break their routine. The least they know the better!

Now, fast forward a few years and I’m telling this story to Development Director, Chris Oakley at a Chinese Restaurant. As the main course arrives those famous words slip out of my mouth that often make Chris put his head in his hands and look angry ‘Chris, it would be really good if PlayoutONE could….’  and then silence. Chris scratched his head, swore a few times and swigged some cider. A pen was requested from the waiter and a few sketches followed on some used napkins. These were to become the blue prints for ‘PlayoutONE Pro Isolation Mode’. So what is it?

Isolation Mode in PlayoutONE Pro essentially takes away all the stresses and strains from the above scenario making it very simple for you to restart the Automation Server whilst staying on the air. When the time comes to restart the main server, the On Air Studio can be pressed into Isolation Mode with one button press. This effectively tells the On Air Machine to run from its own local SQL Server and use six hours of cached audio from its own local AudioStore.

Whilst in Isolation Mode, you’ll see a yellow triangle on the On Air Machine just next to the clock, this acts as a confirmation and warning you that you are now in Isolation Mode. On all other client machines, you will see a red triangle, indicating that the On Air Machine is in Isolation Mode. This is important as once Isolation Mode is enabled, no audio or log changes can be made to the main database. This is because the On Air Machine is now in charge and is constantly syncing back to the on air database what it’s doing, what it’s played etc to ensure no changes or data is lost.

Isolation mode is only designed for short periods of time, and takes away the need for having a CD, Spotify or YouTube. Once the server has restarted, the On Air machine can come out of Isolation Mode, and all changes are synced back to the main database. Warning triangles will disappear allowing you get on with your day!

Hey who knows, ‘Maintenance’ may not be so bad after all!

Find out more on how to setup Isolation Mode in PlayoutONE Pro by reading our setup guide.