AFFECTED SYSTEMS: hq_dia
2023-02-08 : Maintenance Complete
The upstream provider has completed several areas of repair to address the issues outlined here. Several spans of fiber were replaced, and some electronics were changed along the path to address instability. After testing the provider believes the issues to be resolved. Thank you for your patience while addressing this issue.
2023-01-27 : Additional Maintenance Require
The upstream provider will schedule further maintenance to attempt to complete the necessary repairs on the fiber path. Thank you for your patience. Further updates will be provided here when available.
2023-01-24 : Maintenance Scheduled
The upstream provider has scheduled further (and hopefully final) maintenance starting at midnight tonight (00:00 1/25/23) to attempt to complete the necessary repairs on the fiber path. We will update here with the results of tonights maintenance.
2023-01-20 : Further Mainteance Needed
The upstream provider worked throughout the maintenance window, but was unable to complete full repair the issue. More maintenance is required to complete the repair and scheduling is in progress.
2023-01-16 : Further Mainteance Needed
The upstream provider worked throughout the maintenance window, but was unable to complete full repair the issue. A new maintenance window has been scheduled for Tuesday night at midnight (1/18 00:00) to continue work to repair the issue. More updates will be provided here as they become available.
2023-01-12 : 12:10 am : Repair In Progress
The upstream provider has begun maintenance. Brief service interruptions may pop up as maintenance is performed, as traffic switches over to alternate paths and routes reconverge. We will update here later in the morning after the maintenance window closes. Thank you for your patience.
2023-01-11 : Stage 2 Repair Complete
The second stage of repairs were completed on Tuesday night after midnight (Wednesday Morning). The upstream provider advised they will need more time to fully complete repair, and scheduled an additional maintenance window for Wednesday Jan 11 beginning at midnight (1/12 00:00:00). We will post updates here as efforts progress.
2023-01-07 : Stage 1 Repair Complete
The first stage of repairs were completed on Thursday night after midnight (Friday Morning). The upstream provider advised they will need more time to fully complete repair, and scheduled an additional maintenance window for Tuesday Jan 10 beginning at midnight (1/11 00:00:00). We will post updates here as efforts progress.
2023-01-04 : Repair Scheduled
The upstream provider has advised that measures to repair the DIA path with take place on Thursday Jan 5 starting at midnight (00:00 1/6/23).
We continue to notice very brief (<2s>) flaps of this circuit. We appreciate your patience while this issues is resolved.
2023-01-03 : Repair Scheduled
The upstream provider has advised that measures to repair the DIA path with take place on Thursday Jan 5 starting at midnight (00:00 1/6/23). There could be multiple interruptions of service while spans are repaired and equipment is swapped. Thank you for your patience during this issue.
2022-12-31 : Provider Update
The upstream provider has advised they have located the issue, and have engaged management and T3 team to schedule a repair. The path affected serves other customers and USP is coordinating with all involved to schedule maintenance. We will update here when a firm date is set for maintenance.
2022-12-26 : Investigating
We are aware of an issue with one of our DIA Internet connections out of our main datacenter. Engineers are currently investigating. Upstream provider has isolated the issue to a particular piece of hardware near Dallas, TX and is working to resolve.
Details: One of our DIA circuits is experiencing some very quick (less than 2 seconds) flaps, meaning the circuit is going down and back up very quickly. The transition is too quick for the system to fail over to a backup circuit, but just long enough to cause a slight buffer if it happens at the right time.
We expect this to be resolved quickly by the upstream provider. More information will be provided as it is available.
Thank you for your patience and for using NESPARC.
Last updated: January 17, 2024 at 2:37 PM CST