Verizon Users: Still Having Trouble?

I’m so excited! After executive escalation of our ongoing issues with Verizon Wireless users being unable to effectively access the Tripawds forums or their blogs, I finally spoke with a “Verizon Solutions Advisor” who actually acknowledged the problem—and better yet, understands the vzw network node packet loss I first identified more than a month ago.

If my last post seemed a bit technical, that was my point. It apparently got the right people interested, and provided enough information to help pinpoint the problem.*

But fear not! This post will quickly and simply explain how you may be able help get this problem fixed, for good.

How To Help Fix Verizon Network Packet Loss for Tripawds Users

If you are a Verizon Wireless customer, and you have trouble loading pages on the Tripawds website, or logging in to your blog, please help us help Verizon Wireless techs fix this issue by doing the following:

1. Confirm these symptoms when connected to the internet using your Verizon Wireless device:
   1. Visit tripawds.com (Page may load quickly.)
   2. Try navigating to the forums or other pages
   3. Attempt to log in to the forums or your blog
   4. Note any serious delays? Pages take longer than normal to load? If yes, proceed to #2. (If no, thank you for your time.)
2. Find out your current IP address by visiting one of these sites:
   • whatismyipaddress.com
   • whatismyip.com
3. Copy your IP address and send it to us.
   • Use this contact form or email me directly
   • Please include your geographic location
   • If possible, please include telephone number of your Verizon Wireless device

Thank you! Your help is greatly appreciated.

Sadly, I understand that the Verizon Wireless customers this may affect most, might not be able to even read this. Or they may have given up on visiting the Tripawds community weeks ago. Please pass along this information to any members you may know who might have had trouble.

But this can’t be just us! My discussion with our Solutions Advisor confirmed that it isn’t only traffic to our server’s IP address that is getting dropped. It is clearly a block of addresses which means there are certainly other users and/or servers being affected. If you found this post in a desperate attempt to find a resolution, please complete the steps above AND include the IP address of the server you are/were trying to reach, and its physical location, if known.

*And Now for a Bit More Technical Info…

I was right: My mtr reports, ping results, and other network analysis proved that packets are indeed being dropped at a specific node on the Verizon Wireless network. So much for all those “support” reps insisting it was a “problem with the site” for the past six weeks.

I was wrong: The data loss is not being caused by intentional ICMP Rate throttling as I first presumed. So I guess I can’t cry “Net Neutrality!” as I was about to start doing.

During my initial research after identifying the packet loss, I looked into how to diagnose network issues with mtr. When analyzing My Traceroute reports, I discovered the cause could be one of two things:

ICMP Rate limiting — Throttling is a common practice by network providers used to reduce traffic congestion which prioritizes “more important” traffic.

A Misconfigured ISP Router — If a router on the route packets take is incorrectly configured your packets may never reach their destination.

It was our Data Center who first suggested the ICMP throttling. And silly me, I expected Tier 1 Networks like Level3 and Verizon to ensure their equipment handles all traffic effectively. Our Solutions Advisor reassures me that “this isn’t even throttling,” instead telling me that, “something is broken.”

Finally! 🙄

What Now?

Unfortunately, these things are complicated and may take time to resolve. To complicate matters further, even all of the data passed to every server at our data center could be considered a spec in the universe of internet traffic Level3 delivers. It’s not like we’re Comcast with hundreds of thousands of users complaining because they can’t watch the last episode of Mad Men.

We are but a few thousand users with a couple hundred blogs, about three legged cats and dogs.

I’m not getting my hopes up too high, yet. But at least I have someone who has promised to get to the bottom of this.

One more thing…

I  just want to scream at all those support reps who insisted our “site is broken” on so may calls over the past month, I Told You! I know that won’t do any good, but it feels good knowing all of my efforts to escalate this issue have finally resulted in reaching someone who understands.

Before hearing from the Advisor, we finally resorted to purchasing another mobile hotspot device to confirm my theory about the problem being with the network, and not the site, nor the server. Yesterday morning I was totally ready to start asking Verizon Wireless to explain this…

Let’s hope we don’t have to go there, ever again. Please send us you IP address if you have been affected and use the social network buttons below to share this so we can hopefully reach anyone else experiencing the support nightmare I have been living for weeks now.

Please stay tuned for updates, or post in the this forum topic if you care to discuss further!

UPDATE: May 10, 2015

We believe this issue has finally been resolved. Read this post for details.