Documentos de Académico
Documentos de Profesional
Documentos de Cultura
As a result, service level agreements (SLAs) for Ethernet services are incredibly stringent, with severe penalties and
liquidated damages for missing SLA targets. As service bandwidth increases and more critical business applications
are hosted in the cloud, these SLAs will likely get even tougher.
To meet customer expectations, service providers must perform rigorous testing at critical points during the service
lifecycle:
As self-service “bandwidth on demand” becomes more prevalent and SLAs tighten, customer expectations increase.
Furthermore, as service providers virtualize key components of their networks, traditional testing approaches and
standards are no longer fit for purpose.
An innovative approach to testing is needed; one that can help providers overcome service assurance challenges
across the critical stages of the Ethernet services lifecycle with an integrated and automated solution. This solution
must be an integral part of the service delivery chain that evaluates service quality and performance in a way that
emulates, as closely as possible, the customer experience. Implementation of new testing standards such as MEF
48, MEF 46 and RFC 6349 in a service assurance platform can address these needs, ensuring rapid deployment of
services that perform as expected and conform to defined SLAs.
www.spirent.com
The Emergence of Ethernet Service Activation Testing
Ethernet as a service provider delivery technology has evolved dramatically since the introduction
of optical Gigabit Ethernet in 1998. This new technology represented a breakthrough that enabled
service providers to deliver higher speed services to customers at lower cost. With the migration
from TDM to Ethernet services that followed came a need to provide network and service acceptance
testing (SAT) capabilities to ensure that these connectionless services were delivering the desired
performance.
Evolution of Ethernet
1T
400GbE
400G Ethernet Speed
100GbE 200GbE
100G Speed in development
50GbE
Link Speed (b/s)
40G
40GbE 25GbE
10G
10GbE 5GbE
2.5GbE
1G
GbE
100M 10Mb/s
Ethernet
100Mb/s Ethernet
10M
1980 1990 2000 2010 2020
Standard Completed
• focusing on loss as the primary performance metric ignored the impact of excessive jitter on real-time applications,
• the traffic profile of an RFC-2544 did not adequately emulate the variability in frame size and priority seen on
production networks, and
• the device reset and restore tests could not be run in a production service provider network where a single
network element carries traffic for multiple customers.
3
Seizing the Carrier Ethernet Opportunity:
Leveraging New Service Activation Testing Standards and Automated Service Assurance
Ultimately, RFC 2544 was designed to validate the Y.1564 provides a very robust test methodology for
capabilities and limitations of network elements, not as service activation; multiple concurrent streams with
a methodology for testing services, but in the absence differing information rates, frame sizes, and classes
of other standards, it was widely adopted and many of service, evaluation of the ability of the service to
service providers still include an RFC 2544 test in accommodate throughput above the committed
their service activation methodology today. This fact information rate (CIR), evaluation of how well the
notwithstanding, as service providers have enhanced service accommodates burst traffic, the requirement
their offerings to deliver more managed services to measure information rate, delay, jitter, and loss
and fewer simple bandwidth pipes, they needed a simultaneously, and tests to ensure that class of service
similar enhancement to their acceptance test and parameters (VLAN Priority and DSCP) are preserved.
benchmarking toolkit. Y.1564 was a vast improvement over the previous RFC
2544 standard; enabling a more accurate emulation
Test vendors stepped forward with a variety of test of customer traffic. As a result, Y.1564 has been widely
methodologies to address this deficiency, creating adopted by the industry to validate service level
service activation tests that supported multi-stream agreements (SLA).
tests with characteristics that more closely matched
observed traffic profiles and directly measured frame While ITU-T Y.1564 defines an excellent suite of tests for
delay variation (jitter). Given the need for consistency service activation, the industry has lacked a consistent
in metrics needed between service providers, access and agreed-upon procedure for using these tests as
vendors, and customers, the industry coalesced around part of a well-defined acceptance process. This has
a new standard for service activation testing: ITU-T led to inconsistency in metrics and methodologies and
Y.1564. Y.1564 took the basic capabilities of RFC 2544, contributes to delays in service delivery. To address
throughput validation, latency, and burst verification, this gap, the Metro Ethernet Forum (MEF), developed
and enhanced these to meet the needs of an industry the MEF 48 standard to define the testing methods and
which was experiencing a rapid growth in bandwidth results needed to create uniformly understood results.
consumption, real-time protocols and distributed Using the capabilities of ITU-T Y.1564, MEF 48 defines
networks and applications. a standard process for creating a “birth certificate” for
the service which can be used to verify to the customer
that the service is performing per the SLA and can
be referenced in the future (i.e., benchmarked) to
determine if the service performance has degraded.
Because the capabilities of RFC 2544, Y.1564 and MEF
48 are similar in many ways, the table below helps
highlight similarities and differences between these
standards.
www.spirent.com
Table 1: Comparison of SAT testing standards.
5
Seizing the Carrier Ethernet Opportunity:
Leveraging New Service Activation Testing Standards and Automated Service Assurance
Figure 2: TCP-based protocols such as HTTP and HTTPS account for the vast majority of traffic traversing public data centers.
If 90%+ of the customer’s traffic is connection-based TCP traffic, validating only the performance of connectionless
UDP traffic, using RFC-2544 and Y.1564, is not sufficient. Many organizations have used a variety of open source
applications and coopted shareware tools to try and determine TCP performance, but until recently, an agreed upon
methodology for TCP testing did not exist. The publication of RFC 6349—Framework for TCP Throughput Testing
defined a methodology for measuring end-to-end TCP Throughput to provide a better indication of user experience.
Although not an officially ratified standard, RFC 6349 nonetheless defines a series of tests and metrics that can be
used very effectively to characterize TCP performance on the network. By validating key TCP metrics, such as the Path
MTU (Maximum Transmission Unit), Bottleneck Bandwidth, Round-Trip Time in addition to verifying TCP throughput
for multiple simultaneous connections, a clear measurement of TCP performance can be obtained and added to the
service acceptance birth certificate.
www.spirent.com
MEF 46: SAT for Shared / Type II Services
As the volume of managed services increases, so does the prevalence of shared or type II services where the service
delivery involves multiple service providers. This situation, where the last mile is provided by one SP while the end-
to-end service is provided by another, presents significant service activation and fault isolation challenges. Validating
“Off-Net” service performance when the last mile is delivered by a type II vendor has traditionally required a field
dispatch or cumbersome coordination with the vendor. The manual intervention associated with validating these
services directly conflicts with the move to bandwidth on demand and SDN-controlled service delivery.
Type II SAT Session “On Net” Type II SAT Session “Off Net”
The Metro Ethernet Forum (MEF), working with key industry partners has developed the MEF 46 Latching Loopback
specification to enable service providers to perform service activation and fault isolation tests without requiring
management access to or control of ENNI or CPE devices. The capability expands upon the Layer 2 EOAM Loopback
capabilities to implement a true loopback on the far end device that, unlike EOAM, does not alter the reflected frame/
packet. Combined with EOAM, deployment of MEF 46 compliant ENNI and CPE devices in the network enables
Service Activation and fault isolation On Net and Off Net without prior coordination with the Type II Vendor, enabling
dramatic reductions in the time required to activate services and isolate faults. Furthermore, it provides consistent
testing by both the end-to-end service provider and last mile Access Provider. This enables them to collect and
compare the same metrics using identical measurement techniques, virtually eliminating the traditional fault
responsibility and ownership issues that plague Service and Access Providers.
7
Seizing the Carrier Ethernet Opportunity:
Leveraging New Service Activation Testing Standards and Automated Service Assurance
SAT, however, has lagged and many carriers are still using manual procedures to validate services prior to handoff.
This model does not map well to the self-service, bandwidth on-demand delivery models to which service providers
are migrating. Table 2 provides an example of the difference in time required to complete SAT using manual and
fully automated strategies. Automating SAT provides dramatically improved time to revenue while simultaneously
eliminating manual entry errors.
Table 2: Time required for SAT using manual and fully automated test approaches.
Totals (less test execution time): 3 hrs 51 min 41 min 5 min 50 sec
Less obvious is the impact that SDN and bandwidth on demand will have on the frequency of SAT. With the ability
to change service configurations based on their business needs, service performance will need to be validated with
much greater frequency. Addressing this increased demand with manual testing is simply not feasible. Automated
SAT is key to delivering service rapidly and accurately, and automating service acceptance testing requires a fully
integrated test management platform which can execute the desired test strategies in a mechanized fashion with
no human interaction. By necessity, an integrated test solution should include: 1) links to service inventory systems,
knowledge of test resources (Physical Test Heads, 2) Test Functions embedded in network elements, or Virtual Test
Agents), and 3) the ability to compare measured test results against the specific Service Level Agreement.
www.spirent.com
Introducing VisionWorks Transport Service Assurance (TSA) Solution
VisionWorks TSA
Transport Service Assurance
Business Mobile
Ethernet Backhaul
• The only active service assurance solution with virtual test agents and physical probes supporting
up to 100G Ethernet services
• Analytics that integrate both passive network / customer data and active test data
• Extensive library of service activation tests (including MEF 48, MEF 46, RFC 6349, Y.1564, RFC
2544), SLA /performance monitoring tests and more
• Integration to network automation, management and network orchestration (MANO) and SDN
controller functions
9
Seizing the Carrier Ethernet Opportunity:
Leveraging New Service Activation Testing Standards and Automated Service Assurance
A Practical Example
Service providers are attempting to overcome service assurance challenges across three critical stages of the
Ethernet services lifecycle: service activation, performance (SLA) monitoring, and troubleshooting. In this section,
we discuss a practical example of the service provider challenges described in this white paper and describe how
automated assurance can be utilized to solve them.
1: Service Activation
Enterprise Ethernet service activations often involve major logistical complexities which can make them difficult to
get up and running quickly and accurately, leading to reduced customer satisfaction and increased time to revenue.
For example, let’s say a bank based in London wants to open a new office in Frankfurt. The bank approaches its UK-
based service provider to set up a connection from London to Frankfurt. The provider naturally jumps at the chance
and commits to an SLA of activating the new service within seven days.
The bank’s service provider has its own network in the UK and even has another dedicated network in Germany.
However, the provider does not have any connectivity between the UK and German network, and now has to
negotiate with third-party carriers to connect the new high-bandwidth service between its London and Frankfurt
networks. Everything needs to be running smoothly, on all parts of the extended network, in seven days’ time.
As a result, while this provider may commit to very tight service activation targets, they often come up against three
big problems:
• Installation dates don’t get met due to the high levels of project complexity across multiple countries and multiple
providers
• Connectivity is provided on time, but the actual services are not working—and the fault could lie with any of the
various providers involved
• The service delivered doesn’t match what the customer ordered—perhaps misconfigured by one of the third-party
links in the service chain
In addition, the provider’s ability to activate Ethernet services is limited by the number of skilled installers on their
workforce. How do providers scale their expert workforce to meet the increasing demand for Ethernet services?
With VisionWorks TSA, providers don’t have to rely on manual service activation by field technicians. VisionWorks’
service assurance controller and active test agents (i.e., virtual test agents and probes) are deployed within the
network and controlled remotely by an automated workflow or a centralized testing team. By generating synthetic
traffic (such as the activation tests defined by MEF 48) along the connection and using in-band loopbacks (such as the
latching loopback defined by MEF 46) to verify the SLA, VisionWorks TSA enables providers to turn up new Ethernet
& IP / transport services quickly and ensure they’re configured to meet customer requirements.
www.spirent.com
VisionWorks Controller
Backhaul
Internet/IPX
MTSO
ATA
Core Network
ATA Active Test Agent:
■ Virtual Test Agent
Data Center ■ Physical Probe
ATA ATA ATA ATA ATA ■ Embedded Test
Function (NID or
Access Network Element)
ATA
CO
Metro Network Internet/IPX
Core MPLS
2: Troubleshooting
What happens when our bank’s Frankfurt office has a problem getting data from London? The UK-based service
provider gets a call and then has just a few hours to repair the fault or start incurring SLA penalties. With a connection
spanning international borders and serviced by multiple providers, just figuring out where the fault is could take
longer than the mean-time-to-repair (MTTR) window.
Troubleshooting network performance issues in these kinds of complex Ethernet services is difficult and expensive
because:
• Many man-hours are wasted isolating the fault in the first place
• Troubleshooting usually requires multiple dispatches, increasing the cost and time of fixing the fault
• The various providers involved waste time and energy on finger-pointing—while the original service provider pays
out large SLA penalties
Dispatch to Fix, Not Find
Many enterprise Ethernet SLAs have mean-time-to-repair (MTTR) targets measured in hours—and in complex, multi-
country, multi-provider Ethernet connections, these are very challenging targets to meet.
Providers must first mobilize and coordinate multiple teams of technicians just to find the problem—that’s before they
can start fixing it. Meanwhile, the customer is losing productivity, the MTTR clock is ticking and the SLA penalties are
ratcheting up.
With VisionWorks TSA, it takes just a few minutes to isolate the problem remotely. That means providers can quickly
identify the exact root of the problem and only a single dispatch is required to fix the issue. Further, because
VisionWorks TSA integrates with back-office trouble ticketing systems, the procedural delays related to manual ticket
routing and disposition can be automated, further reducing the mean time to repair.
3: Performance Monitoring
With our bank’s London-Frankfurt Ethernet connection set up, the service provider still has plenty of work to do.
Network performance KPIs, including loss, latency, jitter, and availability, must be monitored to ensure SLAs are
being met. And KPI reports must be produced on a monthly or quarterly basis , so the bank can see its services are
meeting expectations.
All service providers need to guarantee specified levels of performance and provide reports to demonstrate they’re
meeting these KPIs. However, providers often find monitoring Ethernet service performance difficult because:
• There’s often little or no visibility into service performance across the network, so providers are forced to take a
reactive, rather than proactive, approach to any issues
• SLAs aren’t met because providers can’t see or respond to the issues early enough
• Providers can’t deliver accurate reports on KPIs, so they end up paying penalties regardless of actual performance
Just as with service activation and troubleshooting, centralized, automated testing capabilities make it far simpler to
monitor and report on network performance.
VisionWorks TSA provides unattended, automated KPI monitoring, enabling a 24/7 view of Ethernet service
quality and generation of accurate SLA reports. VisionWorks TSA also delivers proactive notifications of network
degradation issues, so providers can act before issues affect customers’ services.
www.spirent.com
SAT for Network Functions Virtualization (NFV)
Service and access providers continue to move forward with network function virtualization to increase flexibility
and reduce costs in the network. The widespread deployment of white box compute platforms running virtualized
network functions presents opportunities and challenges with respect to SAT. It has taken decades for Equipment
manufacturers to implement test functionality, including Y.1564 and loopback capabilities in their dedicated switches,
routers, and CPE equipment. As these functions have been virtualized, the focus is on the performance of the primary
function, typically switching or routing packets, and supporting test functions has necessarily been assigned a lower
priority. Further, the performance of virtualized functions typically lags that of dedicated switches and routers, driving
an increased need for service testing.
VisionWorks features a cloud-native architecture that supports physical, virtual and hybrid networks.
However, all the news is not negative. The presence of compute capabilities at various points in the network enables
the instantiation of virtual test agents (VTAs) targeting the specific needs of each service lifecycle stage including
activation, performance monitoring and troubleshooting. The presence of compute resources on consumer premises
equipment (CPE) equipment, for example, would greatly simplify execution of RFC-6349 TCP Throughput tests by
supporting a TCP Client VTA at the customer premise. Once compute resources are present a broad range of testing
becomes possible including application testing, security assessments and other tests not feasible today due to the
limited test capabilities embedded in black box network elements. Another consideration with deploying NFV is
the impact of white box specifications on VNF performance. The time is rapidly approach when validating the NFV
infrastructure (NFVi) must be included as part of a full featured service activation test strategy.
Regardless of the scope of the testing, whether its layer 2, 3 or 4, application layer or infrastructure, the test
capabilities must be designed into the service chain, ensuring that testing is enabled at the same time the service is
provisioned.
Rev C | 08/18