Tuesday, 2 February 2016

Internet Outage (January 19) Report

At 11:21 on Tuesday 19 January the internet connection to the University failed. This left several important University IT services including MOLE and Google Apps inaccessible until our connection was restored at 16:35.

The internet outage affected the University of Sheffield, Sheffield Hallam University, and all the educational institutions in Yorkshire that receive their internet connection via the two Sheffield Universities.

Following the internet outage we had a debrief with JANet (the government funded HE network that provides our connections).

On Sunday 17 January monitoring systems run by the cable provider indicated a ‘low light’ fault causing significantly reduced performance on the back-up link connecting Sheffield Hallam University to the internet. The Hallam link serves as our own back-up internet link. Engineers were scheduled to locate and fix this fault for Tuesday the 19th of January and all internet traffic was diverted to our primary link. From this point onward our internet connection was entirely dependent on the primary link.

At 11:21, road excavations (a man with a jackhammer) severed the primary link and our internet connection was lost.

This double physical loss is a very unlikely event, but clearly our reliance on this connection is critical. We were already investigating the viability of alternative or additional solutions to mitigate against future loss of connectivity.  

This will take longer decide and plan for, and may require us to change the way some services are delivered. The nature of our plans will be communicated once we are clear what we are going to do.