4-day Telus email outage caused by mishap during repair of data storage service

For four days, frustrated Telus customers have reported an outage that has affected the company's email services.

Telus now says the outage happened when the company's data storage cloud service provider, Dell EMC, was repairing failed equipment last Thursday morning and accidentally took telus.net offline.   

They are now working to recover customer data and emails. 

John Zabiuk, an instructor in the applied information systems technology program at the Northern Alberta Institute of Technology in Edmonton, Alta., says being offline this long is unprecedented for a massive company like Telus. 

"That would have to be a pretty catastrophic failure. If they are having to restore customers mailboxes then that means they've lost all of the data basically," Zabiuk said.

The outage affected customers in B.C. and Alberta. 

Restoring data

While some customers had access to email on Saturday, service has been off and on for many. 

Zabiuk says the outage means Telus most likely stored a large portion of its data with Dell EMC. 

"One of the issues of having all of your servers in one cloud provider, and obviously Telus is learning this lesson right now, is that if that cloud provider has an issue, then you've lost everything," Zabiuk said.

"One of the dangerous things with outsourcing all that data to cloud providers that you're reliant on them to safeguard it."

In a statement released on Sunday, Telus said it successfully reinstated access to most customer email inboxes last Friday.

"But [we] later experienced some performance issues as customers re-engaged with the platform," read the statement. 

In the meantime, customers still without email access can use webmail on new servers to send and receive new messages through an internet browser.

Telus says it will take its time in recovering old messages and restoring them to mailboxes in order to not compromise customers' information. The company says this is a complex and time-consuming process, but it is working to ensure an outage like this never happens again. 

"We know that our customers are frustrated, and we are incredibly sorry," reads a company update to clients.

Businesses affected

Hassan El Masri's business, W. Masri Notary Corporation in Vancouver, has had two Telus webmail accounts since 2001. He says they get hundreds of emails every day and do the majority of their work through email.

On Thursday, the business' email accounts went down. As of Saturday, he had some access to his email account, but it has been spotty. 

"We deal with deadlines and financial transfers and legal matters on a daily basis," said El Masri. 

"Because nobody who emails me gets a bounce-back, which is one of the major concerns of this problem, I don't know what is coming in and what isn't coming."

El Masri says communication with Telus has been a problem. He called four times, and tweeted his concerns,  and was told there was nothing Telus could do about re-routing his emails to another account, or making sure clients were getting a bounce-back email telling them he was not getting his mail.

El Masri says his company will most likely switch email service providers. 

Jas Thind, owner of Mix The Bakery in Vancouver, says his business has lost at least $2,000 since telus.net went offline on Thursday. 

"This is very upsetting for the business [because] we don't have communication with our clients," Thind said. 

The Bakery gets most of its catering requests through email. Since he could not get emails on time, Thind lost out on a catering job for the University of British Columbia. 

Thind says what really bothered him was the lack of communication on the part of Telus. 

"As a business we feel they need to operate their services on a better scale. We rely on these companies and they should do their best service."

'We let our customers down'

On Monday, Telus issued a written statement saying they continue to work to restore email access to all users.

Telus chief customer officer Tony Geheran said in the statement that all active customers who have been affected will receive bill credits within the next 48 hours.

"We let our customers down, and this is not the level of service they expect or deserve from us," Geheran wrote in the statement, adding they recognize the importance of personal data, contacts and communications.

"We continue to be incredibly sorry for this disruption and the significant inconvenience our customers have endured."