This paper describes the Space Control Centres use case of the CUBIST project. The readers are not expected to understand the terminology used in the space industry. However, the following three terms are essential:
Telemetry is a common name for technologies that allow remote reporting of information. Telemetry is usually characterized by the rate and the set of parameters. Payload is the useful cargo of a spacecraft, such as a set of scientific instruments and sensors packed in a single assembly. Operator is the person who is constantly monitoring the payload operations.
Operators usually work in shifts and are located in space mission control centres.
The use case covers the processing of the telemetry produced by the SOLAR payload. SOLAR is a set of devices used for Sun observation and mounted on the Columbus module, which is a part of the International Space Stations. SOLAR is operated from the Belgian User Support and Operations Centre (B.USOC) by the operators working for Space Applications Services.
Next to the scientific data, SOLAR generates tens of gigabytes per year of so called "housekeeping telemetry", which contains an overview of the "health" of the SOLAR devices.
In 1998, ESA's Manned Space Program board decided to adopt a decentralized infrastructure for the support of European payloads on-board the International Space Station (ISS). This concept was based on operating multiple User Support and Operation Centres (USOCs), distributed throughout Europe. Each of the USOCs controls one or several payloads. For example, the Biolab payload is operated from the Microgravity User Support Centre (MUSC) in Cologne, Germany. EuTEF and EDR (European Drawer Rack) payloads are operated from the Erasmus USOC in Noordwijk, The Netherlands. SOLAR payload is operated by the Belgian USOC (B.USOC) located in Brussels.
While USOCs are responsible for the payload operations, the Columbus Control Centre (Col-CC) at Oberpfaffenhofen, Germany, has the responsibility of the European Columbus module on board of ISS. Together with the USOCs, the Col-CC coordinates all the European Columbus operations.
Space Applications Services is an independent Belgian space technology company, founded in 1987, whose aim is to develop innovative systems, solutions and products for the aerospace markets as well as related industries. In addition to developing mission control systems, the company also installs, sets up, operates and manages these systems.
Space Applications Services has operations teams at B.USOC and Erasmus USOC. Operations have been ongoing continuously (mainly 24/7) since the launch of the Columbus payload in February 2008. Missions and experiments include Frank De Winne's OdISSea mission, BOP, PromISS-4, SOLAR, PCDF, LES-2, EPO-3, EuTEF, EDR, etc.
SOLAR (see Fig. 1) is an integrated platform accommodating three instruments complementing each other to allow measurements of the solar irradiance throughout virtually the whole electromagnetic spectrum. SOLAR is also one of the first external payloads of the ISS. The SOLAR instruments are mounted on a Coarse Pointing Device (CPD) and make use of the CPD Common Control and Power Distribution Unit (CU) to get power, to collect, packet and dispatch to ground the instruments generated telemetry data and to receive the ground issued telecommands, ISS data and timing synchronization. The CPD accommodates the instruments and provides tracking capability thanks to a two-axis rotating platform and a Sun sensor. The first axis is used to compensate the ISS orbital motion (de-rotation function), while the second axis is used to correct the orbital plane drift and seasonal Sun apparent motion (indexation function). The selected reference position for SOLAR allows the following useful pointing ranges for CPD:
around Y axis: ±40 • for de-rotation from sunrise to sunset around X axis: ±24 • for elevation toward outboard and inboard Three instruments are deployed on the payload: SOVIM (Solar Variable and Irradiance Monitor) measures irradiance in the near-ultraviolet, visible and thermal regions of the spectrum (200 nanometers -100 micrometers). SOLSPEC (SOLar SPECtral Irradiance Measurements) covers the 180 nanometer -3000 nanometer range with high spectral resolution.
SOLACES (SOLar Auto-Calibrating Extreme UV/UV Spectrometers) measures the EUV/UV spectral regime (17 nanometers -220 nanometers) with moderate spectral resolution.
The primary objective of the SOLAR mission on Columbus [1] is the quasicontinuous measurement of the solar irradiance variability with highest possible accuracy. For this reason the total spectral range is recorded simultaneously by the three sets of instruments: SOVIM , SOLSPEC and SOLACES. SOLAR operates on Columbus since February 2008. As of today, the SOLACES and SOL-SPEC instruments are still achieving their scientific objective. SOVIM stopped operating in October 2008 following an electrical malfunction. The 2008-2010 period was marked by an exceptional minimum of the solar spectrum [2] making the SOLAR data the best available reference for solar intensity at the low solar activity.
These data have both scientific and societal importance as the mechanisms of the solar variations are far from being completely understood and as the Sun is the main energy input to the climate system. A historical cold climate period in the late seventeenth century has been simultaneous with a low solar activity known as the Maunder minimum. This importance justifies thus any mean used to study the health of the monitoring instruments and the quality of their data.
The data used in CUBIST is derived from the housekeeping telemetry stream of SOLAR. Next to the science data, housekeeping telemetry is the biggest in size, amounting to many tens of gigabytes per year [3].
SOLAR has been operational for more than three years, already, sending one telemetry packet every second or so. Over a year, this represents approximately 3 • 10 7 packets. Each telemetry packet contains 343 parameters. 44 parameters do not change at all or very rarely. Among the others, 135 have binary readings, such as ON and OFF. Others have readings that span between 3 and 2 • 10 6 distinct values.
The data released to CUBIST consortium partners covers 30 days between September 26, 2008 and October 25, 2008. 4 This period has been selected because a major event occurred on the 25th of October, at 04.28 AM. The DC/DC converter powering SOVIM, one of the three instruments used by SOLAR, broke down.
As CUBIST unrolls, additional data sets may be considered for conceptual analysis in the project.
In order to properly monitor the health of SOLAR payload, the B.USOC operators have access to a part of the SOLAR telemetry, the housekeeping data. This telemetry is organized into packets that are sent by the payload to the control centre. It contains readings and variables, such as temperatures, voltage and current readings, operational states and reports from all different aspects of the payload. We provide below an overview of the different parameters sent downstream in the housekeeping telemetry.
Temperatures Although the SOLAR thermal control keeps the platform and the instruments within their operational boundaries, many temperature sensors allow the operators to closely monitor the temperatures. These temperature readings are of the type float and are available for the Control Unit, the Power Boards (PB1 and the slave boards PB2 and PB3), the motors, the three instruments and the Sun Sensor. The temperature limits are defined in the mission database and, although thermal control should insure these will not be crossed, they will be flagged when the limit is near to be reached. Besides these hard limits which could damage the hardware, soft operational limits are also defined for the scientific measurements. For example, the SOLACES instrument can only perform science measurements within the temperature range of 17 − 20 • C. These limits are currently not set within the mission database, but are common knowledge of the operator.
Power supply Housekeeping of the different power boards of SOLAR is also available. The main power board (PB1) powers the CU and the slave boards (PB2) and (PB3). The later power the motors and thus allow the tracking of the platform and the instruments. For each of the boards, the status of the board and its outlets, temperatures, DC/DC converters, voltages and current are available. This housekeeping data is closely checked during power-on activities and continuously monitored during operations. The status parameters are defined as strings (ON/OFF) and readings are of the float type. For some, an operating range is defined.
Instrument housekeeping For each of the instruments, a set of housekeeping data and their range with respect to the operations, has been defined by the scientists. Besides those, the SOLAR CU also provides the status (ON/OFF) of the instruments, temperature readings and communication status (OK,NOK) in the telemetry. Additionally, for the Sun Sensor, the sun presence reading is also included in the downstream data.
Pointing Device telemetry To support the actual SOLAR operations, and to execute science measurements, an additional set of parameters is made available to the operator. These parameters are related to the status of the platform, the movements and the sun observation.
Based on ancillary data of the ISS, containing the station's attitude, SOLAR software calculates the start time of the sun observation, the duration, the indexation and de-rotation angles for both axes, next midnight and noon, and the observation counter. These integer parameters will then trigger the actual Sun tracking of the platform when they are within the platform mechanical and structural range. Supporting the actual movement, the status of the platform such as the Zero_Proc_Flag and the On_Target_flag indicate whether the pointing device is calibrated and motor controlled, allowing proper Sun tracking.
Another important parameter is the SOLARMode. This string parameter indicates whether the operator can define software settings (SCM mode) or update the Software (SMM Mode), whether SOLAR can perform Sun Tracking (PM Mode and submodes) or in case of an anomalous situation (SBM).
System variables SOLAR allows the operators to change some system variables and to declare them to the housekeeping in the so-called User Selected Data area (USD). A total of 12 USD are available and each have a specified type (integer, float, string). For the on-going operations, around 6 are used.
Several steps [4] have been taken to reduce the size of the data set, in order to make it easier to run FCA analysis on it:
-The period from the start of September 26, 2008 to midnight of October 25, 2008 has been extracted. -A list of 208 parameters that are relevant for the operations has been established. Other parameters have been filtered out. -Nominal limits for parameters of type float have been adjusted by an operator to reflect realistic operational limits. All the float parameters have been replaced by the textual labels NOMINAL, WARNING and DANGER, signifying that the value is within operational limits, in the warning zone or is dangerously abnormal. -In order to reduce the cardinality of the data, readings of the moving axes of the SOLAR payload have been averaged to 1 • .
Today, in space control centre operations, much time is spent on transferring, discussing, reviewing, and copying information between the operations partners. Also the search and replay of operational data in order to be able to correctly analyse the on-board situation is time consuming. This problem is especially emphasised by the fact that many of the data stores are distributed and have different user interfaces. The current operations show an increasing need for a system providing the operator with adequate and fast information and analytics, especially during anomalous situations. A system federating and managing the broad amount and variety of available operations data, from Console logs to House Keeping Path Telemetry where the operator would benefit the accessibility to other related data in a transparent way. In case of an anomalous situation the operator would have the possibility to immediately identify the possible failure and to propose a way forward or work around, mitigating the science loss. This section lists use case scenarios that are studies by CUBIST. Each anomaly discussed below has its roots in the real operations. Depending on the anomaly, a particular research strategy for CUBIST has been identified.
SOLAR Science measurements are often performed through so-called command schedules. A command schedule is a dedicated pre-programmed time-ordered sequence of time tagged commands to be sent to the SOLAR instruments or the CPD system On various occasions the B.USOC operator encountered a unexpected event during a run of a command schedule, from a sudden stop of the script to the anomalous behaviour of the instruments. The analysis of the failure that follows is often restricted to that particular command schedule. The CUBIST project could allow the operator to trace back the command itself inside the script that was sent and presumably generated the error and even back track in the archive whether this particular command has been sent before, independent of the command schedule. With a user-friendly form, the operator can check previous events and the reaction of the payload on it. This will benefit the failure analysis in finding the actual cause of the anomaly, rather than the environmental circumstances.
It may happen that the payload manifests an unforeseen thermal situation. That is, a situation when the temperature of one or several sensors changes in an unusual way, albeit within the nominal limits. The operator is then charged with finding similar occurrences inside the telemetry archive and with the determination of typical thermal and power profiles. Currently, the search in the telemetry archive is usually done as a real-time replay of the telemetry archive. A more intelligent solution that employs the results of the concept analysis may be implemented, so that an automated agent finds occurrences of similar situations by taking into account the telemetry parameters.
Since the start of the SOLAR operations SOLAR experienced on a regular basis a reset of the internal Analogue Interface Board, causing the platform to go in the anomalous Stand-by Mode and halting all on-going science measurements. Resuming science can only be done by power cycling the payload. The occurrence of the anomaly seems rather random, but due to the amount of SOLAR parameters this has not been confirmed. The CUBIST project will correlate the occurrences of these failures with other parameters and might reveal a pattern with SOLAR parameters or even with factors outside the payload. This would allow the operator to act proactively and avoid a hard stop of the on-going science measurements. In the case of operational instruments, frequently a recovery procedure may not be practiced after the commissioning period and presents a risk if it is improvised. So an orderly power cycle, fully documented, will always present less risk than a general shutdown proceeding from an undetermined state.
A few months after the launch of the SOLAR payload, SOVIM, one of its three scientific instruments died because of an electric failure in a DC/DC converter. It is still unknown whether this failure could have been predicted given the previous telemetry stream. The objective of the CUBIST system would be to find patterns of failure in the flow of telemetry parameters with the aim to transpose these to the prediction of future failures.
Parts of this work have been carried out in CUBIST (Combining and Uniting Business Intelligence with Semantic Technologies) project. CUBIST is funded by the European Commission under the 7th Framework Programme of ICT, topic 4.3: Intelligent Information Management.
The information in this document is provided "as is", and no guarantee or warranty is given that the information is fit for any particular purpose. The above referenced consortium members shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials subject to any liability which is mandatory due to applicable law.