ESA's spacecraft and payload operations support networks are facing rapidly increasing demands for network bandwidth, induced by the requirements for the Columbus Orbital Facility and by Earth-observation projects like Envisat. The traditional implementation of operational networks is based on technologies that are adequate for lower data rates, but which become very expensive when rates of tens of megabits per second have to be supported. Since the traditional technologies permanently reserve the bandwidth of the maximum data rate of bursty traffic, these resources generate costs even during trafficless intervals. The new Asynchronous Transfer Mode (ATM) technology has the potential to respond to such shortcomings in today's systems and to meet tomorrow's communications requirements in a very cost-efficient way.
In its preparation of a ground infrastructure to support the future operation of the Columbus Orbital Facility (COF) and the Automated Transfer Vehicle (ATV), ESA has already defined a communications network that will exploit the benefits of Asynchronous Transfer Mode (ATM). To validate the applicability of ATM technology for meeting the various service requirements, ESOC, in cooperation with Deutsche Telekom, has set up an ATM Testbed, which in the initial phase will serve to prove the concept. After the implementation of the operational networks, this Testbed will support simulations, trouble-shooting, and compatibility testing for new equipment before it is moved to the operational network. The ATM Testbed has been integrated into an existing communications reference facility which has already been used as a prototype implementation in supporting telescience missions with Spacelab and the Mir space station.
Today, the telephone network has become our most efficient communications tool because it combines global availability, short response time, and user friendliness. Consequently, the newer services like digital facsimile still exploit the telephone infrastructure. However, as the analogue telephone network was not designed to support these digital services, other dedicated public and private data networks have been set up. Each of these networks has needed its own resources. As a result, the equipment involved has been too specialised for each service to make efficient use of the spare resources of another network.
Nowadays, new communication services are appearing, with sometimes as yet unknown exact requirements. Examples are video conferencing, high-speed data transfer, videophones, video broadcasting, and home learning (tele-education).
In order to support old, new, and future (unspecified) services in a unified manner, a Broadband Integrated Services Digital Network (B-ISDN) has been envisaged. As only one common network needs to be designed, the overall costs of design, installation, operation, and maintenance should be reduced. ATM has been chosen as the transmission mode for this integrated network. This technology is connection-oriented, switched (like the telephone network, where there is a process of Ôcall setup' and a line connects the two telephone sets for the duration of the conversation), and multiplexed (like the data networks, allowing capacity sharing between different users and applications). The circuit set up for the duration of a call is therefore virtual, rather than real.
ATM has been designed to be service-independent, thereby allowing the integration of voice, data, image, video, and multimedia traffic over the same network. Such a network will be flexible enough to adapt itself to new needs and anticipated services. When establishing a connection, the user is allowed to specify an Ôexpected category of service'. The five ATM service categories represent new service building-blocks that make it possible for users to select specific combinations of traffic and performance parameters (Quality of Service, or QoS), thereby providing a much greater degree of flexibility and fairness in the network's utilisation. Figure 1 shows how these service categories, which are briefly described below, share the capacity of a link:
Figure 1. Qualitative overview of link usage by different service categories
For example, two-way voice communications have tightly constrained delay requirements, so that a CBR service is required. On the other hand, for an overnight data-distribution service, delay is not a key issue, and the capacity of the network can be used on an availability basis; the ABR service would be used in this case.
In this way, customers can choose to access the network how and when they actually need it, while still maintaining their specific efficiency and quality requirements. Network operators can achieve maximum use of the deployed resources by sharing them between all customers and fulfilling the different user needs in a cost-effective way. The customer expects to profit from appropriate tariff strategies (e.g. the use of spare network bandwidth on an as-available basis should be priced considerably lower than the use of guaranteed bandwidth).
One of the candidates for an ATM-based implementation is the network to support COF and ATV operations (Fig. 2). Known as the Interconnection Ground Subnetwork (IGS), it will connect all European sites to NASA in the USA, RSA in Russia, and the NASDA network in Japan, for remote operations and scientific- payload support.
Figure 2. COF and ATV target scenario, including the optional JEM link
The primary inter-orbit link between the COF module of the International Space Station (ISS) and the COF ground segment is provided via NASA's TDRSS satellite relay system, and that on the ground via the corresponding NASA facilities at Johnson Space Center (JSC) in Houston and Marshall Spaceflight Center (MSFC) in Huntsville on the American side and ESOC on the European side. A second path between the COF module, via the Japanese Experiment Module (JEM) and the European (Artemis) and Japanese data-relay satellites to European and Japanese ground facilities is being investigated as an option.
The main facilities within the ESA Member States are the control facilities for COF and ATV, the ESA Astronauts Centre (EAC), the User Support and Operations Centres (USOCs), and the engineering support facilities for COF and ATV. ATM-based nodes will be installed at all sites that communicate via the IGS. A node typically consists of one or two 19-inch racks which accommodate the communications equipment. The network is operated centrally from a dedicated Network Management Facility (NMF) located at ESOC. All elements of the network, including the nodes at the most distant sites, are remotely monitored and configured by the operator at ESOC utilising the dedicated NMF capabilities.
The IGS is required to support a broad spectrum of communication services, ranging from typical data-distribution services for telemetry, telecommanding and mission management information services, to new high-speed data applications for science, multimedia applications like video conferencing and multicasting, and voice conferencing for operations co-ordination. For experiment operations, a significantly large bandwidth and possibly highly bursty traffic will have to be supported, with data rates up to 32 Mbit/s. In addition, the high-rate data flow for experiments is unidirectional from MSFC to ESOC and from there to the scientific users. Compared with this data flow, the bandwidth requirements in the reverse direction are rather modest. This type of traffic can be mapped perfectly into an ATM implementation, taking advantage of potential network cost savings for this type of traffic profile. In ATM, asymmetric and bursty traffic allows efficient sharing of link resources with other users. While the traditional technologies charge for permanent, bi-directional, symmetrically available bandwidth, with ATM the data transport service should ideally be charged according to the data volume transported. In practice, this is not quite attainable as, for example, requirements for real-time data services may impose less economical service conditions.
Other new projects like Envisat also have very similar communications requirements in terms of data volume and asymmetric traffic flow, where ATM again offers the most cost-effective solution. As explained above, ATM services are characterised by a variety of QoS classes with different tariffs. The choice is driven by the requirements in terms of reliability, data quality, delay tolerance and priority. As the mapping of requirements to service implementations is performance- and cost-sensitive and the carrier providers (telecom operators) are reluctant to provide pricing and performance prognoses for the years 2000 and beyond, the ATM Testbed provides an efficient platform with which to gain the necessary technical experience.
ESOC's ATM Testbed is one element of a larger communications reference facility, which evaluates networking and communication techniques for ESA projects before they are actually proposed as implementation solutions.
There is a major trend in the implementation of ground segments for ESA projects to build on commercial off-the-shelf (COTS) equipment and services. This approach has also been followed in the field of networking, i.e. data transport between the relevant sites involved in mission operations, for many years. For communication systems located at ground stations, control centres and other facilities involved in mission operations, this trend is still evolving. This means that COTS products are used wherever possible and enhanced as necessary with custom developments to meet the requirements of individual missions.
The communications reference facility serves as a test environment, into which technical solutions based on COTS products and services are integrated and then tested for their suitability to meet specific mission requirements in terms of functionality, bandwidth guarantees and availability (fail-safe scenarios). An important aspect of the test environment is that it is used to verify not only integrated products, but also the services being offered by international telecommunications operators. In the real-life implementation for each mission, the overall system ultimately relies on international communications services, e.g. traditional leased lines, Integrated Services Digital Network (ISDN), Frame Relay Virtual Private Network (VPN) service, ATM and Very Small Aperture Terminal (VSAT) based services. It is therefore crucial, particularly for new services like Frame Relay, ATM or VSAT, to validate technical interfaces and service performances in close cooperation with the carriers providing these services on an international basis.
The ultimate goal of the communications reference facility is to validate technical solutions that meet ESA's project requirements in terms of both performance and cost, and which can be implemented within a well-defined schedule, thereby enabling a surprise-free implementation approach.
Since ATM-based services are well suited for high-rate data transport, high rate being defined here as above 2 Mbit/s, and delivery of high volumes of data, high volume being defined as more than 10 Gigabytes per day, special attention is being devoted to emerging ATM technologies and services. Hence the need for the ATM Testbed (Fig. 3), which has three main elements: equipment related to the ATM services and the private network, the access to the public ATM infrastructure, and the legacy services and applications which need to be integrated.
Figure 3. ATM Testbed configuration
The two ATM switches are access concentrators and allow for the integration of all of the legacy services (shown in the shaded area) within the ATM network. They are linked together through different transmission media to create a private ATM network. The work stations and the routers act as native ATM users of these network on their ATM interfaces. From both switches public-carrier access is available, allowing the testing of interoperability and of the re-routing recovery mechanisms in the case of a (simulated) failure in the private network.
The shaded area corresponds to the existing legacy services - both data and circuit services - which should be integrated within the ATM network. For the data services, Frame Relay and Ethernet interfaces are available. For circuit services, like videoconferencing, voice, and data, over the existing (proprietary) switching system, the Circuit Emulation service is the most suitable.
An ATM network analyser makes it possible to monitor the data traffic and performance and to take measurements within the network. With a de-facto standard user interface, the analyser allows different parameters to be measured during the course of the tests. Figure 4 shows a snapshot of the analyser. The screen displays a Traffic Simulation tool, which allows one to inject up to three different kinds of traffic into the ATM network. Each traffic source can be modelled using mathematical statistical distributions.
Figure 4. The ATM internetwork analyser
The validation concept used in the ATM Testbed follows the principle of local testing as much as possible without interfacing to carrier services, in order to minimise the costs involved. The two ATM nodes are therefore first connected back-to-back in a local setup by a variety of emulated leased lines (E1: 2Mbit/s, E3: 34 Mbit/s, and STM-1: 155 Mbit/s). In this configuration, all ATM services provided in the current software version of the node are verified. Individual tests always involve end systems and/or special test equipment that allow the generation of data in the required manner to stress the nodes, e.g. to allow for the occurrence of congestion in the switches in order to observe their exact behaviour. This is illustrated in Figure 5. The middle rack carries two pieces of switching equipment, with the routers below and the work station with the management software in the front part. The analyser is used to inject the traffic into the network to stress and observe the behaviour of the switches.
Figure 5. The ATM test setup at ESOC
Only after the satisfactory completion of these first tests are the nodes connected via a national public ATM network (T-Net ATM from Deutsche Telekom). In this configuration all ATM services provided by the national network are verified for interoperability with the private ATM equipment. These tests provide a first indication of what can be reliably achieved in a real networking scenario that involves only one carrier.
Figure 6 is an example of such a test. The network analyser is connected to the access port of the public network, and a loop-back is performed in a far-end ATM switch in Cologne (Germany), shown on the right, to measure the performances over the different interfaces.
Figure 6. Test via the ATM network of Deutsche Telekom
When this set of tests has been performed satisfactorily (with the limitations observed), the nodes are internationally linked via interconnected carrier ATM networks. In this configuration, all ATM services provided by the carriers involved are verified for their interoperability with the private equipment. Results of these tests in general show further limitations in terms of available ATM services and constraints on their usage due to the interoperation of different carrier networks. These tests give a first indication of what can be reliably achieved in a real networking scenario that involves multiple carriers. Such a scenario has to be very close to the implementation options which have to be considered for ESA missions.
Future envisaged test scenarios could be the transport of high data volumes from Kiruna in Sweden to ESOC in Germany and to ESRIN in Italy for Earth-observation missions, and the connectivity from MSFC (USA) to ESOC in preparation for providing the communications support for the COF.
The tariff structures of ATM networks distinguish one-time installation charges, regular monthly charges and variable-usage charges. The one-time charge is for the installation of the service. The monthly service charges are based on the access circuit speed and the distance to the next ATM cross-connect switch. This is typically a leased line between an ATM user site and the next ATM node of the telecommunications network provider. The variable-usage charges within the ATM network are composed of the connection charges based on requested bandwidth, connect time of day, connection duration, connection distance or traffic-volume-dependent charges.
ESA's future spacecraft and payload operations will require substantial data transfers between receiving ground stations, ESA sites, the sites of other space agencies, and the user sites (control centres, science user sites, etc.). The operations concept distinguishes on-line telemetry and telecommand services for time-critical real-time monitoring and control, and offline telemetry services for non-time-critical measurement data. The bulk of the data is not time-critical.
Therefore a potential utilisation scenario could be such that the crucial time-critical online services, which are only used during satellite connect times (passes over ground stations), will be supported by appropriate variable bit-rate services. The less time-critical offline services could be supported by available bit-rate services, making most efficient use of available excess capacity in the carrier network. In a private-network implementation, these types of services would be supported over the same physical links. For the critical variable-bit-rate services, the traffic contract would specify the sustainable cell rate, while for the less time-critical available bit-rate services the traffic contract would specify a minimum cell rate. In other words, the communication system would automatically distinguish between the two types of traffic, where the bulk data applications would always be treated as background tasks making use of the available excess capacity in the network without requiring special network-management intervention.
With such a utilisation concept, ESA can profit substantially from the forthcoming network services and their different tariffs. This holds true for the case where ESA makes use of commercial services, but also for the case where ESA is the network operator (private network approach). In the latter case, the new network services will lead to higher network utilisation, due to dynamic bandwidth sharing with guaranteed throughput for the critical online applications.
Telecommunications technology is changing at an unprecedented rate. Since a newly emerging technology like ATM lacks a proven performance record under real operational conditions, the establishment of a testbed infrastructure is a mandatory step in order to minimise the risk inherent in its introduction into a mission operations environment and to demonstrate its overall viability.
Such an approach also has other inherent advantages. Space projects can be made aware of the benefits of this communications technology, especially the cost-saving that it brings for the transfer of large volumes of data to the scientific user community. Distributed data applications, normally developed by other engineering teams, can be designed to take into account this network technology, and their interoperability can already be validated with the testbed infrastructure during the design phase. The complexity of the network configuration and operations can be assessed, and personnel can be trained well in advance of the operational phase.
Finally, the testbed experience provides the means to be better prepared for providing realistic implementation plans for ESA's future space projects, and for benefitting to the maximum possible extent from the availability of the latest ATM services.
The authors gratefully acknowledge the continuous support that they have received from the ESA Directorate of Manned Space Flight and Microgravity, which made much of the work reported here possible.