A global tier 1 Service Provider (SP) was experiencing a common issue: its circuit inventory data was untrustworthy, so it could not rapidly isolate faults and lacked the granular network intelligence to build out capacity, when and where it was needed. The SP had tried the common solution of periodically purchasing costly blanket bandwidth upgrades but still encountered the symptoms.
An end-to-end service orchestration solution enabled the SP to effectively and quickly determine the root cause of network degradations and outages. By gaining a real-time and accurate view of its network topology and circuit paths, this SP has already seen the frequency of truck rolls go down by more than a third, and forecasts in the long term that the improved network efficiencies will drive up OPEX and CAPEX savings.
Fire fighting faults in the network: The challenge
This SP was experiencing typical daily network outages but they were having difficulty sectionalizing where the on-net or off-net issues were occurring, and determining the root causes of the degradations.
Disparate fault, performance and utilization systems made the Network Operations Center (NOC) technicians unable to isolate fault and services degradation in real-time in the Ethernet mobile backhaul network. Adding to the challenge, they were forced to “swivel-chair” between the numerous legacy systems in place including standard Operations Support Systems (OSS), a capacity utilization system, and a Radio Access Network (RAN) RF performance measurement system. In their NOC, the technicians had a schematic diagram that showed all the circuits, but this view could not display fault or performance data in real time. They did not have an application that could visualize the entire backhaul circuit from the aggregation router to the macro or small cell at the end of the path. To resolve one issue, technicians had to look at multiple applications on multiple screens to try to correlate the issues manually.
The SP urgently needed an approach that would enable them to effectively correlate fault, service Operations and Maintenance (OAM) and utilization statistics via a single end-to-end view of the network. This would enable them to quickly determine the root cause of network degradations and outages, and become more precise in their capacity planning for bandwidth upgrades.
Troubleshooting the network: The solution
Being able to visualise and analyse their network circuit data in real-time was key for the SP when choosing a solution. Of huge appeal to this SP was the ability of the Ethernet Lifecycle Manager (ELM) from CENX to perform continuous, real-time, data audit of information from the disparate maintenance systems, thereby enabling an operator to rapidly isolate and use one-click testing to troubleshoot faults from a single view. The SP determined that ELM’s Service Visualisation module was the only solution that enabled a view of the entire circuit, including off-net segments, microwave hops, and Ethernet over SONET.
Another major factor in choosing ELM was its Network Analytics module that allows for a precise forecast of capacity hotspots and the ability to determine CAPEX investment timing with far more precision. ELM provides the SP with exact details on average bandwidth utilization, peak and burst utilization and enables the SP to forecast when an Ethernet Virtual Circuit (EVC) will become a given percentage full (i.e., 70%) - so they need only upgrade EVCs if and when needed. Costly and wasteful blanket upgrades would be in the past.
This SP also chose ELM’s Service Level Agreement (SLA) Management module for its extensive SLA performance monitoring and reporting of SP and alternative access vendor (AAV) network segments, and more effectively automates the increasingly complex SLA process in order to provide an optimal customer experience across multiple carrier domains.
End-to-end Service Orchestration automates the lifecycle management of Carrier Ethernet and IP services across today’s data network infrastructures, as well as evolving Software Defined Networking (SDN) and Network Functions Virtualization (NFV) networks, to deliver demonstrable business benefits: clear ROI due to increased revenue through accelerated time-to-market, and significant cost efficiencies from faster and more accurate fault, performance, capacity and SLA management.
Joining up the network: The deployment
The SP had millions of dollars invested in existing systems, which didn’t require replacement since the CENX Professional Services team deployed and integrated ELM into the legacy infrastructure. Unlike other vendors, CENX increases the utility of legacy systems without costly rip and replace.
The process of deployment began with a CENX-facilitated discovery workshop with the SP’s network operations team. The resulting business case showed that ELM would solve the challenges the SP was experiencing due to the disparate legacy systems, and it quantified the potential ROI benefits to be gained by CAPEX and OPEX savings.
Today the SP’s NOC is no longer inhibited by disparate legacy systems, but has a central accurate repository of the Ethernet backhaul inventory and topology and a single dashboard to visualise, monitor and troubleshoot the overall network right down to each backhaul circuit, router, cell site and on-net/off-net segment.
The real-time troubleshooting module isolates faults to specific network segments, automatically identifies the problem owner, and uses one-click testing to validate and troubleshoot issues. In figures, this has meant a 25% reduction in time to triage and repair service degradations and outages, which improves customer experience and reduces OPEX. There has been a 40% reduction in the cost of cleaning inventory data, a 40% reduction in dispatch and truck roll frequency and costs, and the SLA management process is 70% more efficient and authoritative.
The future roadmap looks positive for this SP as ELM will support the transition to SDN by bridging its existing legacy systems with newer SDN-controlled systems. OPEX savings through efficiencies of scale will also accelerate as the number of monitored sites grows from 50k to 70k and beyond.