A Samsung Galaxy 8, Phantom Notifications, and @Tmobile’s Dreadful Support

This is a blog of two topics. The first, a brief technical explanation of a problem with my Samsung phone after an upgrade to Android 8.0 (Oreo) pushed by T-Mobile, the subsequent debugging, and hopefully help for anyone else experiencing the issue. The second, my horrible experience with T-Mobile Twitter-based tech support.


On April 2, T-Mobile pushed an over-the-air update for my Samsung Galaxy 8 (G8) phone. In addition to a routine Android security patch level update, it also upgraded the phone to Android version 8, code-named Oreo. Shortly after the update, I started getting what I called ‘phantom notifications’, between one and six of them every hour or less. These were audible notifications that didn’t correspond with any discernible event on the phone, sometimes in quick succession. Over the course of a week, there were a few times where an icon would appear in the notification bar for a split second, making me think it was related to a specific event, but I couldn’t figure out what. I engaged with T-Mobile on Twitter, and they offered some ideas. Here is everything I did to debug and figure this out, based on their questions and my own ideas.

  • T-Mobile: SMS App Clear Data/Cache (I suspected it may be related to SMS)
  • Me: Full power cycle
  • Me: Changed default notification to determine if the phantoms are using system notification preferences (they are)
  • T-Mobile: Verify Notification Reminder functionality = OFF
  • T-Mobile: Verify no wireless/bluetooth/NFC turned on during phantoms
  • T-Mobile: Clear cache partition on phone via Debug menu
  • Verified software versions for all functionality (‘About Device’)
  • T-Mobile: Verify all apps are updated via play store
  • T-Mobile: Verify no apps from unknown sources
  • T-Mobile: Enable Developer options (did not change anything)
  • T-Mobile: Device Maintenance showed no app crashes, no hint of a problem
  • T-Mobile / Me: Phantom notifications do NOT vibrate, while SMS is configured to (so not SMS)
  • T-Mobile: No SD card in phone
  • T-Mobile: Uninstall Samsung Health (they suspected app causing this, that app isn’t on the phone)
  • T-Mobile: Backup SMS and clear all of the messages
  • Me: DND mode suppresses the phantom notifications (observation)
  • T-Mobile: Confirm I did not download ANY new apps on Sunday (day before update), Monday (day of update), or Tue – Thur (after update)
  • T-Mobile: Confirm the last time my phone worked w/o phantom notifications was Sunday and Monday before the patch (and every day prior since buying the phone)
  • Me: twice out of hundreds of times, i have seen a ‘health monitor’ type icon appear in notifications for a split second when it happens
  • Me: One-by-one disable app notifications, wait for phantom. process of elimination = found the offending app = PROBLEM SOLVED

Naturally, it was the last app on the list I had notifications enabled for. “Weather & Clock Widget for Android” by Devexpert.NET, which worked fine on Android 7.x, started causing these phantom notifications on Android 8.0. Uninstalling and re-installing did not fix it. The only reason I had allowed notifications from this app, is it would put the current temperature in the notification bar at all times. Blocking notifications for this app didn’t allow this behavior, but also stopped the phantom notifications. No factory reset needed.


Part 2; My dreadful experience with @Tmobile tech support via Twitter DM.

First, this isn’t the first time I have Tweeted and had them reach out via DM, offering support. I don’t recall having a good experience with them before, and this time certainly takes the cake on a poor experience. I am writing this up as a warning to others who might go this route, and as feedback to T-Mobile so they better understand what it is like on the customer side, and offer some tips for improving.

Perhaps the biggest problem with T-Mobile Twitter support, is their system for interacting with customers appears to be designed to resolve issues very quickly. I can’t speak to their workload, average customer engagement time, etc. But for a case like mine? I went through 22 different people over the course of seven days. On April 8, there were nine different people that cycled through to ‘help’ me. On April 7, while working with Reggie (who happened to be the only one out of 21 that I felt was truly helpful), he said he needed to AFK for 15 minutes for break, implying that someone else would take over. By that point, I knew I had already gone through seven others, so I told him I would happily wait until he returned. This high turnover rate on support staff worked against the process entirely for my case. Each time, the new person had to try to read the thread and figure out what was going on, and they rarely skimmed the thread it seemed. When I was offered a summary of my problem by the new person, it was typically wrong or left out important bits. T-Mobile needs to better identify problems that can’t be solved in ten minutes, and keep one or a few people on the case for consistency. When a customer repeatedly asks for a specific support person to re-engage, listen to them. Here is the list of people I dealt with:

  • Apr 3 – Joel Bannister
  • Apr 3 – Harley Sumida
  • Apr 3 – Ruben Hernandez
  • Apr 3 – Dee Medina
  • Apr 3 – Zach Ricketts
  • Apr 3 – Kimmi Smith
  • Apr 3 – Victor Loya
  • Apr 7 – Reggie Reese
  • Apr 7 – Harley Sumida
  • Apr 8 – Lauren Chan
  • Apr 8 – Pete Harman
  • Apr 8 – Marva Biggar
  • Apr 8 – Sora Yi
  • Apr 8 – Marva Biggar
  • Apr 8 – Kate Tomallo
  • Apr 8 – Lauren Chan
  • Apr 8 – Meghan Parks
  • Apr 8 – Eddie Gough
  • Apr 8 – Scott Degelman
  • Apr 8 – Ray Butler
  • Apr 9 – Dee Medina
  • Apr 9 – Mike Perez
  • Apr 9 – Alex Kimbrell
  • Apr 9 – Zach Ricketts
  • Apr 10 – PoxMaphixat [1]
  • Apr 10 – Kyle Saragosa
  • Apr 10 – Scott Degelman

[1] This was the only person that didn’t appear in Twitter DMs with a real name shown by Twitter:


The next bigger problem I faced, is that T-Mobile’s documentation for their support staff is out of date. It’s as if they had never debugged an issue on a Galaxy 8, despite them selling it for half a year. During the ordeal of figuring out my problem, I ran into several times where support failed related to this:

  • Apr 3 – Document for changing SMS message sounds is outdated, not correct for G8 (you apparently can’t on this model)
  • Apr 3 – T-Mobile said to set up a notification log for debugging purposes, yet G8 removed that functionality (ridiculous)
  • Apr 7 – The location of the ‘build number’ to enter developer mode is different on the G8 than previous models
  • Apr 7 – They asked me to go to the ‘Security’ screen in options, yet on the G8 that is ‘Lock Screen and Security’
  • Apr 7 – T-Mobile diagnostic data said ‘apps from unknown sources’ was enabled, my screen said it was disabled
  • Apr 8 – They asked me to check the ‘Samsung Health’ app (there is none, apparently part of the ‘Activity Zone’ app, but that function is disabled)
  • Apr 9 – T-Mobile kept telling me a factory reset is the way to fix this, despite it not necessarily working
  • Apr 10 – T-Mobile told me a factory reset is the way to go AFTER I solved the problem (WTF?!)

After having to correct the T-Mobile support staff this many times, and figure out how to find what they were looking for, it shows an obvious gap in their support ability. As someone who wrote my fair share of technical documentation, I cannot stress how important this is.

As mentioned above, when a new support person steps in, they have to skim the thread to catch up. One person told me that they take extensive notes to alleviate that problem, but after most of the new people offering me a summary got major parts wrong, I don’t think that is the case. Even if they do take notes, I think they are not consolidated, not done in a way for easy transition of the case, and generally convoluted. This causes the support staff to repeat the same things, ask the same questions, and waste customer time.

Next, T-Mobile needs to make sure their employees understand policy. Compare:

  • Apr 3 (Vinny) – “Thanks a bunch for remaining engaged with us at T-Force today, my name is Vinny and I’ll be taking over from here, as Krystn, as she had to step away.”
  • Apr 3 (Joel) – “Thank you so much for reaching out to T-Force! My name is Joel and I will be your #MagentaExpert!”
  • Apr 3 (Ruben) – “I hope you are having an amazing day. My name is Ruben and I will be taking excellent care of you and all of your concerns/questions today.”
  • Apr 3 (Zach) – “Thanks for sticking with us here. My name is Zach, and I’ll be taking over from here.”
  • Apr 7 (Reggie) – “I do want to introduce myself, my name is Reggie and I will be your #MagentaExpert today.”
  • Apr 8 (Meghan) – “My teammate had to step out for a quick meeting but my name is Meghan and I’ll be taking over to provide you with excellent service!”
  • Apr 8 (Eddie) – “Fun fact, Since T-Force is a team and constantly changes to ensure that customers always have support 24/7 we are not supposed to share our name since it already shows on the message.”

After support staff introduced themselves by name six times, Eddie came along and said they aren’t supposed to share their name. He further points out that Twitter shows their name (in the native web interface, not in Tweetdeck BTW), and yet that isn’t the case either as seen by “PoxMaphixat” above.

While some that interact with T-Mobile may say they are really ‘nice’, to me, that isn’t the case. Their overboard attempts to portray a fun and friendly atmosphere are insulting and a waste of time. Throughout the week, I was assured that they were there to help and resolve my issue, while not reading the prior messages, not understanding the issue, and bouncing in and out of my ticket to the point it was difficult keeping up with them. The phrase they loved to over-use, “I will be your #MagentaExpert!” is a joke. Seven days to figure out my problem, and they never did, I had to. Other phrases they love to say, adding fluff and not actual support, while not reading the thread and repeating the same things over and over:

  • I absolutely want to be able to help you in any way that I can!
  • It’s great seeing you here today. I hope you are having an amazing day.
  • That is an awesome question and definitely not something I am familiar with, but we can definitely work together to look into it!
  • I honestly want the best and fastest resolution for you!
  • Thank you for taking time out of your day on this!
  • Here at T-Force, we value customers time and always want to get them the best resolution possible without wasting their time.
  • We’ve got your back! (T-Mobile needs to remove this from their playbook, it is insulting.)
  • I really appreciate you reaching out and working with T-Force today.

Overall, I need a lot less of this fluffy wording, and a lot more I didn’t quote, and more actual support. If you have to keep telling me you “have my back” and want to give me the “best resolution possible”, you are convincing me you aren’t good at your job. We expect customer support to do that already.

Apr 3 (Joel) – “If you prefer to not do that, then you always have the option to back up the device and reset the software completely.”
Apr 3 (Zach) – “Can you please tell me if you’ve completed a master reset on the device since the update?”
Apr 3 (me) – “If a ‘master rest’ means a ‘factory reset’, that may be a deal breaker.”
Apr 3 (Zach) – “Typically, if there are any bugs that come across after an update, which this one may just be, a factory reset would be the best possible solution, as inconvenient as it can be to set everything up again.”
Apr 3 (Kimmi) – “In those instances the only fix I’ve been able to locate based on user feedback is a factory reset of the device.”
Apr 3 (Kimmi) – “Unfortunately the only option we have at this time is to complete the reset.”
Apr 3 (Victor) – “The master reset would be a great way to fix the issue in case it’s just some sort of temporary issue. ”
Apr 7 (Reggie) – “By no means do I want to tell you that you absolutely must do this, but in the end I want to respect your time and I feel like at this point the Master reset might fix the issue permanently whereas what we have done has demonstrably had no effect on the issue at hand.”
Apr 7 (me) – “If a factory reset is the answer, then I walk from Tmobile and go on a social media campaign to dissuade people from using Tmobile, because that is just sloppy programming and a complete breakdown of tech / customer support.”
Apr 8 (Marva) – “I know Reggie mentioned a master reset and that seems to be the only thing we haven’t tried up until this point, is that correct?”
Apr 8 (me) – “Safe mode has not been tried, and a reset, the nuclear option, is out of the question.”
Apr 8 (Sora) – ” I know that you do not want to do a master reset … I totally follow your logic; I do want to mention that if the software update is giving this error, then a master reset does allow the software to be restored on your phone properly.”
Apr 8 (Marva) – “The next step in troubleshooting is to complete that master reset.”
Apr 8 (Kate) – “The Master Reset sounds nuclear, but truly is the faster and cleanest resolution available.”
Apr 8 (me) – “As I said earlier this week, a factory reset means I will no longer be a T-Mobile customer, and will blog about this entire mess, that T-Mobile sent faulty software and could not debug it, and now is pressuring me to go that route while ignoring my direct questions about Samsung Health buginess, that icon that shows sometimes, and my desire to explore that route. That said, do you still think a factory reset is the right option instead of pursuing valid leads that may fix this without a reset?”
Apr 8 (me) – “From there, process of elimination can tell likely tell us which app is causing them. No safe mode, no factory reset. Please add this to your CS playbook.”
Apr 8 (Eddie) – “With the awesome software that we have nowadays, a master reset is the best option since there’s a high chance the bug will be deleted, and your information will be downloaded onto your phone within less than one hour if it’s backed up”
Apr 8 (me) – “Ugh, STOP. Do not recommend a factory reset to me again. I just gave a viable option to better figure this out that will take a few hours, and you go back to factory reset, after I have REPEATEDLY said that is a nuclear option and I a) will not do it OR b) do it and no longer be a tmobile customer.”
Apr 8 (Eddie) – “I just wanted to assure you that we are going to be here for you until we get a resolution. Never wanted to tell you that you should do a master reset.”
Apr 8 (me) – “I mentioned I found a new solution to this kind of problem, to add to your play book. And you immediately recommend a factory reset despite me REPEATEDLY saying ‘no’. You understand no means no right? I am tired of being told why a master reset is the option, and I am *more* tired of Tmobile reps not reading why it is NOT necessarily the right option, why it is NOT a guarantee it will fix anything.”
Apr 9 (Alex) – “If so, have you installed them and reinstalled them? Those are the first two steps, so let me know how that goes!”
Apr 9 (me) – “Two? There were *19* people on the Tmobile side during the course of this investigation, all of who gave up and told me to factory reset.”
Apr 9 (Alex) – “Now, I know we mentioned a master reset was something we should try.”
Apr 9 (me) – “Pretty much confirmed, “Weather & Clock Widget for Android” by http://Devexpert.NET is the one causing the phantom notifications. Uninstalling and re-installing it to start.”
Apr 9 (me) – “Uninstall & Reinstall did not fix it. So there is some weird issue between the app and the Oreo update. I can get around this by disabling notifications for that app, which only makes it so I don’t get the temperature in my notification bar. With that, I have figured it out after 6 days, and without a factory reset, which half a dozen or more of your agents kept telling me to do, over and over and over…”
Apr 9 (me) – “I also explicitly said last night to STOP telling me to factory reset.”
Apr 9 (me) – “I have asked half a dozen times and every single one of you jerks ignore me. Focus on THAT problem instead of a factory reset.”
Apr 9 (me) – “With that, I have figured it out after 6 days, and without a factory reset, which half a dozen or more of your agents kept telling me to do, over and over and over…”
Apr 9 (me) – “At this point i am 99.99% sure I have this resolved, again, without a factory reset.”
Apr 10 (PoxMaphixat) – “Resetting the device and processing a warranty exchange is our last resort. Which would result in a device that is fully reset as well. This might be the thing we would need to do since we’re not able to resolve this phantom issue.”
Apr 10 (me) – “Not only have i solved the issue, I have said repeatedly NOT to recommend a factory reset to me, and you assholes keep doing it. NO MEANS NO.”
Apr 10 (Kyle) – “We can see that you’ve invested a lot of time with these issues on your phone and wanted to avoid going through the previous steps that’s you’ve already done, which is why we were looking at the master reset as a last resort … So our troubleshooting steps would basically be the master reset as well though I Samsung may have more support on what’s going on with this app.”
Apr 10 (me) – “Seriously? You suggest a master reset AGAIN when I have said over and over NOT to tell me that? I solved the phantom notification issue without a reset,”
Apr 10 (Kyle) – “I would reach out to Samsung as I completely understand your concern regarding the reset and they would be able to support the app even further. Does this make sense, Brian?”
Apr 10 (me) – “You said ‘reset’ again. How can I be any more clear here? Never, EVER, not a single time, EVER tell me to factory reset my device. Don’t even mention the word ‘reset’, let alone ‘master reset’ or ‘factory reset’. I honestly feel like there is a den of rapists and molesters working at Tmobile, who don’t understand what the word ‘NO’ means. Does this make sense, Joel / Harley / Ruben / Dee / Zach / Kimmi / Victor / Lauren / Pete / Marva / Kate / Meghan / Eddie / Scott / Ray / Mike / Alex / Zach / PoxMaphixat / Kyle?”

After this? Scott said ‘reset’ once more shortly after my last message. This is the text-book definition of the worst customer support that can be offered. A customer specifically says, over and over, not to recommend a bad support option (the factory reset). Yet, T-Mobile kept recommending it every single time. It gets to the point where it is a trigger word for me, because it clearly shows the support person didn’t read the prior messages. It means that the support staff didn’t leave a message for the next person not to bring up a factory reset. Worse? I SOLVED the technical issue, without a factory reset, and said as much. T-Mobile’s solution? Keep recommending a factory reset anyway, when it was clearly not needed. This is hands-down the worst customer service you could possibly offer, and completely insensitive to a customer. I don’t really care where the breakdown happened, other than it happened half a dozen times, but when a customer says “do not do $thing“, you should NOT do $thing. No questions, no arguments, no equivocation. Yet T-Mobile ignored that basic point, that basic understanding of the tenets of customer support. 18 separate times, reset was their answer, three times after resolving my issue.

My next advice for T-Mobile is to embrace an old classic of customer service. Over six days, interacting with 21 different support people, after repeated complaints about many of them, no manager stepped in. At least, no one identified themselves as a manager, no one exhibited any signs they were a manager, and absolutely no one made it a point to get me a resolution other than the empty “we’ve got your back” lies. Imagine going into a Taco Bell and talking with 21 employees trying to resolve a problem, that your Mexican Pizza was missing ingredients or not cooked, and that entire time no manager stepping in to ensure you got a properly prepared and cooked food item. To me, the customer, those scenarios are no different.

Finally, the bigger picture. I engaged support for one problem, the phantom notifications, which I eventually resolved myself. During the process, T-Mobile asked me questions that highlighted other problems. Despite figuring out the original, I left the engagement with two additional problems that they did not resolve. First, I asked how to disabled ‘Bixby’ completely, and they couldn’t help. Like so many other things, they didn’t understand the software, and/or their documentation wasn’t updated. I had to tell them to disable it per their instructions, it required creating a Samsung account. You actually can’t access the real settings of that malware without creating an account. That is atrocious and just bad design. Second, when we went down the road of the occasional phantom notification icon that I saw, it led us to the ‘Samsung Health’ feature within ‘Activity Zone’. On my phone, it says “tap here to get started” and tapping there does nothing. T-Mobile never helped with that, and after specifically asking them to half a dozen times, they told me to talk to Samsung.

Two more bonus observations, that came up during this ordeal. First, the T-Mobile software update downloaded over 4G, not WiFi. It used to prompt you if you wanted to wait for WiFi and this time it did not. Second, I mentioned that T-Mobile was still sending SMS notifications to me before 9 AM, and one of the support people were gung-ho saying that was not right, they would take my complaint to the top! Well, good luck there, since the last time I brought that issue up on Twitter it did go to the top, all the way to the office of the executives. Nothing ever came of it and I still get text messages from them before 9 AM. If you are going to grab that flag and head on a crusade on my behalf? Maybe consider better helping fix my original problem first.

So, T-Mobile, I have given you a wide variety of ideas for improving customer support. It is in the context of a support case you can easily reference. These ideas are very much in line with many other support services offered by similar services and companies. It’s time for you to up your game.

Advertisements

T-Mobile’s Poor Implementation Works Against Amber Alerts

Just over a month ago, I received a pop-up alert on my Samsung Galaxy 3 (via T-Mobile) with a standard, and persistent, emergency broadcast noise…

Emergency alert
Longmont, CO AMBER Alert: LIC/245FLJ (CO) 2001 Blue Ford F350 Pickup truck
Type: AMBER Alert

The noise stopped briefly, then picked back up again until I tapped “OK”. This is a radical departure from the previous product behavior and service provided. Presumably this came with the latest Android update T-Mobile pushed shortly before (May 13).

No warning about this change, no indication where the alerts are coming from, no explanation on criteria for receiving (Longmont is almost 40 miles north of me, outside a metropolis of ~ 4.5 million), no indication of how often we receive them, a repeating noise that we have to acknowledge (as opposed to SMS that gives a noise/vibration one time only), etc. I’m not opposed to getting such warnings but I should be able to opt in and control the settings for how it is displayed.

One hour later, I received the same alert. That is intrusive and annoying. When it happened, I thought “if this shit happened at night, it would wake me up and force me to get up to ack the alert and turn off the phone” and just that happened. Wednesday early morning, at 5:20AM I received another. As I thought, it woke me, given the emergency sound and vibrating on my desk.

amber_alert

Looking at the SMS options that control this is also interesting. I now have to receive “Presidential Alerts” and cannot opt out of them. There are also Imminent Extreme alerts, Imminent Serious alerts, and the Amber alerts that I have received twice now. What are the others, and what differentiates them? When was the last time a Presidential broadcast was sent to everyone’s email address or home phone number? Absurd you say, why is it all of a sudden OK to send them to every subscriber’s cell phone?

What bothers me the most is that the Amber alerts, and presumably the others, do not adhere to the rest of my SMS settings. When I get an SMS, it vibrates once, makes an audible noise of my choice once, and sits idle until I check the phone. Amber alerts come up with a different sound; one that repeats until I acknowledge it.

Screenshot_2013-05-25-17-28-55

This is ridiculous. I want to receive them, but on my terms. The current setup and being woken at five in the morning forced me to disable the Amber alerts. T-mobile’s crappy technical implementation has worked contrary to their intentions by annoying customers into disabling them. This works against the entire purpose of having the alerts pushed via cell phones.

T-Mobile SMS Disclosure

Yesterday while waiting for a friend to arrive for a movie, I got a curious text from an unknown number with a 337 area code, saying that I had the wrong number. Since I had not called or texted that number, I replied as such. A few texts later, the stranger sent a screenshot of their phone showing that I did send them messages.

Looking at my sent messages, the messages received by my friend, and the screenshot from the kind stranger, I figured out what happened to some degree.

I sent a string of text messages to my friend. She received most, but not all of them. For whatever reason on the T-Mobile side, they decided to send a few of my messages to the stranger. Note that my friend and the stranger do not have similar numbers, and that they are in different area codes even.

Screenshot_2013-06-04-16-59-20

What is odd is that the stranger got a few of them, but not all, not even all of the ones sent in the same ~ 5 minute period.

Screenshot_2013-06-04-16-59-11

Further, my friend got one of the messages that also went to the stranger. So not only were some mixed up and sent to a stranger, one went to multiple people. It’s pretty clear that this was a one-off situation, but it makes me wonder what happened, if it happened to other people, and/or how widespread it was. The obvious implication of this issue is that a sensitive SMS redirected to an arbitrary person could be embarrassing to say the least.

Figured I would document it here for posterity and just in case it happens again to someone else, they can hopefully find prior incidents.