SANY concrete results

Design Principles

SANY Sensor Service Architecture relies on following design principles:

Rigorous Definition and Use of Concepts and Standards

The SensorSA makes rigorous use of proven concepts and standards in order to decrease dependence on vendor-specific solutions. This helps to ensure the openness of a sensor service network and support the evolutionary development process.

Loosely Coupled Components

The SensorSA allows the components involved in a sensor service network to be loosely coupled, in which case loose coupling implies the use of mediation to permit existing components to be interconnected without changes.



Technology Independence

The SensorSA is independent of technologies, their cycles and their changes, as far as practically feasible. Accordingly it is possible to accommodate changes in technology (e.g. lifecycle of middleware technology) without changing the SensorSA itself. The SensorSA is independent of specific implementation technologies (e.g. middleware,
programming language, operating system).

Evolutionary Development – Design for Change

The SensorSA is designed to evolve, i.e. it shall be possible to develop and deploy the system in an evolutionary way. The SensorSA is able to cope with changes of user requirements, system requirements, organisational structures, information flows and information types in the source systems.

Component Architecture Independence

The SensorSA is designed in a way that service network and source systems (i.e. existing information systems, sensors and sensor networks) are architecturally decoupled. The SensorSA does not impose any architectural
patterns on source systems for the purpose of having them collaborate in a service network, and no source system shall impose architectural patterns on a SensorSA. Important here is that a source system is seen as a black
box, i.e. no assumptions about its inner structure are made when designing a service network.

Generic Infrastructure

The SensorSA services are independent of the application domain, i.e. they can be used across different thematic domains and in different organisational contexts. Ideally, any update of integrated components (e.g. sensors, applications, systems, ontologies) requires no or only little changes to the users of the SensorSA services.

Functional Domains

Services in the SensorSA are designed to support applications that serve the needs of users. They may call other services if this is required to fulfil the functions offered at their interfaces. In an extended situation, chains of service operation calls may be defined in order to realize more complex functionality. In a service network every service instance may call operations of any other service. The Service Viewpoint of the SensorSA categorises service types into functional domains expressing the area of concern for which they are basically designed:

• Services in the Sensor Domain cope with the configuration and management of individual sensors and their organization in sensor networks. They are abstractions from the proprietary mechanisms and protocols of sensor networks. An example is a take-over service in case of an imminent sensor battery failure.
  
• Services in the Acquisition Domain deal with access to observations gathered by sensors. This includes other components in a sensor network, e.g. a database or a model that may offer their information in the same way, i.e. as observations. The information acquisition process may be organised in a hierarchical fashion by means of intermediate instances, e.g. with data loggers
  
• Services in the Mediation and Processing Domain are specified independently of the fact that the information may stem from a source system of type ‘sensor’. They mediate access from the application domain
  to the underlying information sources. They provide generic or thematic processing capabilities such as fusion of information, the management of models and the access to model results. In addition, support for service
  discovery, sensor planning and the management of events and alerts are grouped in this domain.

• Services in the Application Domain support the rendering of information in the form of maps, diagrams and reports such that they may be presented to the user in the user domain.
• The functionality of the User Domain is to support the interface to the end user, typically in a graphical fashion.

Implementation Platforms

The SensorSA concepts are realised following the guidelines and technologies of standard (Web) service platforms. However, there are competing Web service paradigms on the market with disparate protocol bindings (e.g. SOAP or HTTP) and capability descriptions (e.g. service-oriented or resource-oriented). The technologies for these service platforms rely upon specifications of W3C, OGC and OASIS.

In order to enable service interoperability, the SensorSA separates the platform specification into a core mandatory part and one or more optional platform specific parts. Three platforms are currently supported: W3C Web Services, OGC Web Services and so called RESTful Web Services for the resource-oriented approach.

The SensorSA core mandatory part refers to W3C Web Services which is, according to a decision of the OGC Technical Committee, also the strategic direction for all new specified OGC services. It requires a SOAP envelope embedded into an HTTP message as the transport protocol and WSDL as the service description language. Optionally, HTTP may also be used directly. This enables to also use the OGC Web services as they are specified today.

Optional, RESTful Web Service interfaces rely upon the principle of Representational State Transfer (REST) which means, that the call of service operation is considered as a transfer of state information of uniquely identifiable resources in form of resource representations. In the SensorSA, the resources are typically geospatial resources described in a resource model, e.g. a collection of sensor observations with a known geo-location reference, and their representations, which may be maps, tables or diagrams.

The multi-platform approach of the SensorSA facilitates the reuse and integration of existing software components and the evaluation of other service paradigms.

Interaction Patterns

SensorSA supports two main interaction patterns:

Request/Reply Interaction Pattern

In a classical request/reply interaction pattern, the consumer ‘knows’ the provider and initiates the interaction by "requesting" the information through direct call to a service interface. This interaction pattern is always synchronous, and the consumer has to keep listening until receiving the reply.

Event Driven Interaction Pattern

In "event driven" interaction, the consumer and the producer are linked through broker. In order to access the information, the consumer first declares interest in getting event notifications by issuing a subscribe operation with an event filter to a notification broker, usually together with a callback address. Independently of this action, the provider publishes events to the broker, e.g. in case of a state change of an underlying resource. The provider is unaware of the consumers that have subscribed to events. It is the task of the broker to analyse the event filters and to determine which consumer should be asynchronously informed by the broker about the event happening.

A-priory, any observation in SensorSA network may be communicated either in event-driven manner, or using the request/reply interaction pattern. Due to practical concerns, the event driven interaction pattern is primarily used in situations where request/reply interactions can not be used , or does not perform adequately. "Event-driven" use cases therefore often include some form of alerting, e.g. "new service/resource available", or "house on fire". However, the choice of interaction pattern may be also dictated by business reasons, e.g. because network configuration does not allow direct access to resources, or because the inexpensive hardware (e.g. smart sensor nodes) can not support relatively resources-hungry web services.

Service Interfaces

SensorSA service model considers service interfaces to be the unit of reusability on specification level whereby an interface is structured into operations. Operations access underlying data sets which are often related to attributes of features defined in application schemas according to the general feature model

Service types are specified in terms of one or more interfaces, whereby one interface may be attached to several service specifications. For instance, the meta-information of services, their so-called capabilities, is specified in a dedicated capabilities interface which is common to all SensorSA services. Supported service interfaces include:

Basic Interface Types	Enable a common architectural approach for all architecture services, e.g. for the capabilities of service instances
Annotation Service	Relates textual terms to elements of an ontology (e.g. concepts, properties, instances).
Catalogue Service	Ability to publish, query and retrieve descriptive information (metainformation) for resources of any type. Extends the OGC Catalogue Service by additional interfaces for catalogue cascade management and ontology-based query expansion.
Feature Access Service	Selection, creation, update and deletion of features available in a service network. Corresponds to the OGC WFS but is extensible by schema mapping.
Identity Managementand Authentication Service	Creates and maintains identities. Supports the management of groups (of identities) as a special kind of identity. Proves the genuineness of identities using a set of given credentials and issues session information.
Map and Diagram Service	Enables geographic clients to interactively visualize geographic and statistical data in maps (such as the OGC Web Map Service) or diagrams.
Ontology Access Interface	Supports the storage, retrieval, and deletion of ontologies as well as providing a high-level view on ontologies.
Policy Enforcement Service	Handles authentication and sends authorisation requests to the Policy Decision Point for non-security enabled Web services.
Policy Management and Authorisation Service	Provides a decision on whether some identity (e.g. a user or a service) is authorised to access a certain resource.
Profile Management Service	Creates and maintains (user) profiles and their associations to identities.
User Management Service	Creates and maintains subjects (users or software components) including groups (of principals) as a special kind of subjects.
Web Processing Service (WPS)	Start, stop and result retrieval of information processes (e.g. statistical calculations).
Sensor Observation Service (SOS)	Provides uniform access to observations from sensors and sensor systems that is consistent for all sensor types including remote, in-situ, fixed and mobile sensors.
Sensor Alert Service (SAS)	Provides a means to register for and to receive sensor alert messages.
Sensor Planning Service (SPS)	Provides a standard interface to task any kind of sensor to retrieve collection assets.
Web Notification Service (WNS)	Service by which a client may conduct asynchronous dialogues (message interchanges) with one or more other services.
OASIS Web Service Notification interfaces	Family of related interfaces that define a standard Web services approach to notification using a topic-based publish/subscribe pattern.

Uncertainity and Quality Assurance

Within the SensorSA, the uncertainty of data sets is described using the UncertML. UncertML allows the information modeller to describe the uncertainty of a specific data set in an interchangeable way using an XML document conforming to the UncertML schema. This XML document can be embedded in a SensorML document to express information about the uncertainty of some process. In addition, UncertML can also be embedded in an O&M document to express the uncertainty of a specific sensor observation.

All data in SensorSA has an associated uncertainty depending on the available meta-information on how the data was observed (measured) or derived from other data sources.

Uncertainity of Measurements

In the case of data measured by sensors, the uncertainty can be classified into two categories:

• Type A: uncertainty arising from a random effect; evaluated by statistical methods, e.g. by computing the standard deviation of the mean of a series of independent observations or by analysing the variance and random effects in data in dependence of experimental parameters.

• Type B: uncertainty arising from a systematic effect. A typical cause of Type B uncertainty is is measurement bias due to the calibration of the measurement instrument or its behaviour in given environmental conditions (e.g. temperature, air pressure), or over time (deterioration of instrument, measurement drift). It is evaluated based on information about the instrument and environment. The measurement values may be corrected to compensate for known systematic effects.

Uncertainity of Processing

In addition to data arising from sensor measurements and observations, SensorSA also handles the results of various processing algorithms, such as spatial and temporal interpolations, or predictions arising from plume modeling. The results of such data processing steps are themselves uncertain, on the one hand due to the uncertainty of the
input data, on the other hand due to the probabilistic or approximate nature of the processing itself. The uncertainty of data arising from processing is typically expressed with one of the following:

• Probability density function, e.g. a normal distribution with known mean and variance. The data value would then lie within one standard deviation of the mean with probability 68% and within two standard deviations with probability 95%.

• Intervals (the data value lies in [a,b]). This does not a-priori assume a uniform distribution on this interval; this would however be the case if the distribution of maximum entropy were chosen. An important special case is when then the measurement instrument can assert that the data value is below or above a given threshold, but can provide no further information.
  
• Statistics such as standard deviation and moments, or quantiles (the data value lies in [a,b] with probability 95%)

Quality Assurance

In many environmental domains, the results of the measurements, observations or processing are subjected to one or more automatic, semi-automatized and manual "quality assurance" procedures. In the course of data quality assurance, the original data is usually annotated, and parts of the data may be declared invalid, or even corrected. The resulting "quality assured" data typically has lower uncertainty than the raw data,

SensorSA and OGC SWE foresee mechanisms for presenting the annotated data, e.g. by mean of the O&M "composite phenomena", but the quality assurance process itself and the type of annotation is domain dependent and therefore can not be fully specified on a generic level. In SANY, the feasibility of SensorSA-compliant quality assurance has been demonstrated within the Air Quality (SP4) pilot application.

OGC SWE Services

Sensor Web Enablement is a standardisation effort driven by the Open Geospatial Consortium. Through its liaison with ISO TC211, the OGC is closely involved in the development of often legally binding services published by ISO.
To date, OGC has successfully established its Observation and Measurement specification as a new work item in ISO. In the future, other SWE standards will follow. SensorSA inherits following SWE services and data models:

• Sensor Model Language (SensorML) – defines a data model and encoding to describe processes and processing components associated with the measurement and post-measurement transformation of sensor observations.
  
• Observations and Measurements (O&M) – defines a data model and schema for encoding measurements and observations.
  
• Sensor Observation Service (SOS) – defines a service model and interface encoding for the provision of sensor measurements and observations, from simple sensors to complex sensor systems, both physical and virtual.
  
• Sensor Planning Service (SPS) – defines a service model and interface encoding for the execution of sensor tasking and parameterization requests.
  
• Sensor Alert Service (SAS) – defines a service model and interface encoding that enables subscription for and notification of situations of interest based upon continuous evaluation of incoming sensor observation streams.
  
• Web Notification Service (WNS) – defines a service model and interface encoding for distributing incoming information to registered users via various communication protocols. It is often used for supporting asynchronous communication and routing urgent messages to whole groups of users according to their communication preferences.

Consequently, all existing SWE-compliant software components can be used in SensorSA Networks. In SANY, the reference implementations of the OGC SWE services developed by 52 North were used both "as is", and as the basis for own developments (e.g. within Cascading SOS). In addition, SANY partners also developed several prototype SWE services with SOAP and REST bindings.

Map and Diagram Service

Have you ever felt the need for more graphic functionalities in a WMS? The "Map and Diagram Service" (MDS) specifications define extensions to the OGC WMS and SLD standards with cartographic features such as diagrams, patterns and custom symbols expressed in Scalable Vector Graphics.

Diagram and other proportional symbol overlays are geo-referenced, and (technically) indistinguishable from regular WMS layers. As a consequence, all WMS client applications are compatible and can be used for viewing thematic layers that can be overlayed on regular cartographic materials. However, the creation of thematic maps on the fly requires dedicated MDS clients that are able to use the extended diagram parameters.

MDS defines several types of diagrams, including pie charts, bar charts, and line charts. Contours (see figure) are another important feature for the effective and intuitive representation of sensor data and interpolated fusion results.

A reference implementation of the Map and Diagram Service specifications is being implemented in the "QGIS map server", which is offered under GPL licence and available for download at:

Download: QGIS Server home page

The development of this service started in ORCHESTRA IP , and continued in SANY IP . The result is an open source application which is mature enough to compete with existing WMS implementations. If you need an application capable of cartographic symbolization while remaining compatible with existing WMS clients, then this is the application you have been looking for. Nevertheless, the usual disclaimer "this application is provided as is, with no express or implied warranty..." applies.

SensorSA Catalogue Service

Catalogue services play an important role for the discovery of resources. Conventional catalogues usually contain meta-information about available data and service resources. A typical user query to a conventional catalogue could include ‘give me all services supporting standard interface x’ or ‘give me all datasets in a specific region, where the responsible party is y’.

A catalogue used for the discovery of sensor related meta-information needs to address additional requirements. Typical queries for such a catalogue differ from the conventional ones. Some examples may be: ‘give me all 'temperature' observations in Austria of May 2009’ or ‘give me all entries supporting a specific sensor type’. Looking at these queries it is clear that additional search criteria and specific meta-information are needed, which reflect the needs from the sensor domain.

SANY addressed these challenges in developing a meta-information schema for the catalogue which follows the Observation and Measurement Model (O&M) from the OGC (Cox). This model is used by Sensor Observation Services, which provide the meta-information necessary to answer the queries above. Besides conventional catalogue resource types (data and service) SANY defined the following new meta-information resource types according to the O&M Model for the catalogue:

• The ‘Feature of Interest’ representing the observation target
  
• The ‘Observed Property’ describing the phenomenon to be observed (e.g. temperature)
  
• The ‘Procedure’ representing a specific sensor, sensor system(s) or algorithm(s) used by a system.

SensorSA Security Framework

As an open architecture, SensorSA does not specify what any particular sensor or service does to protect itself. What the SensorSA does include, are security provisions to control access to services that are considered part of the SensorSA. The focus of the Security Framework is on access control. In a nutshell, access to a particular service is controlled in accordance with a policy specified for that service.

SensorSA security framework uses the SAML tickets to define Identities (individual users), Roles (attributes of Identities, indicating their function - e.g. "administrator" role), and Groups (sets of Identities). The Access Control Policies are specified using (Geo)XACML XML dialect.

The SensorSA Security Framework provides the software components that manage policies and identities, and enforce the policy rules. This includes:

• The Identity Management & Authentication Service is responsible for the management of identities, their authentication, and the management of credentials and issuing of sessions. An instance of the Identity Management and Authentication Service acts as both authentication provider and identity provider. The service supports the management of groups (of identities) as a special kind of identity.
  
• The Policy Management and Authorisation Service supports the management of policies, acting as policy administration point by allowing the management (select, create, update, delete) of (Geo)XACML policies,
  as well as policy information point. Moreover, as an instance of the authorisation service interface it acts as policy decision point by providing a decision on whether some identity (e.g. a user or a service) is authorised to access a certain resource.

• The Policy Enforcement Service handles the necessary interaction (authentication and authorisation) to obtain the required access control decision and is independent of the controlled service (generic).
  
• The Service Proxy mimics the controlled service and delegates the service request to the Policy Enforcement Service.
  
• In addition to the services supporting the Service Access Control Pattern the Profile Management Service manages profiles and their relations to identities.

SensorSA security framework is published under GPL, and can be downloaded from the "Downloads" section of the SANY-IP web site.

Time Series Toolbox

AIT decided to continue the TS-Toolbox development after the project end, and publish most of the components and several example applications under terms of the GPL. Please follow this link for more information.

The Time Series Toolbox (TS Toolbox) is a set of software components and application programming interfaces that simplify the task of building applications that record, process, store and publish time series of observations.

In SANY the TS Toolbox components are used in the following applications: the Cascading SOS Service, the SensorSA Data Acquisition System, Generic Time Viewer, and in the Universal Data Pump – a simple application that provides a convenient way for transporting data from one application, service or file for which a TS-Toolbox data connector exists to another.

The TS Toolbox contains software components for the following functional areas:

• Data connector components implementing access to data using various protocols and data formats
  
• Core components interfacing with the connector components and providing specific additional functionalities like data processing or caching
  
• Frontend components implementing interface functionality (user interfaces or software interfaces)

The functionalities implemented by TS Toolbox components provide application developers with higher-level building blocks than typical general purpose libraries, and allow rapid development of fully fledged applications. The TS Toolbox also includes example applications that can be either used as they are, or as a basis for developing more complex applications. At a time of publishing this information, the TS-Toolbox included following components:

Cascading SOS

The Cascading SOS software developed by AIT is distributed under terms of the GPL, as a part of the Time Series Toolkit, . More information on Time Series Toolkit and Cascading SOS can be found in the Downloads section of SANY-IP.

Cascading SOS (SOS-X) is a client to underlying SOS service(s), and provides alternative access routes to users (or services) interested in accessing the data. In its simplest form, cascading SOS replicates part of the data offered by underlying SOS service with no changes to the data model. This kind of service replication can be e.g. used to assure controlled access only to a subset of data offered by lower-level SOS, without relying on complex security mechanisms. More complex use cases include:

• "Custom View to Data"
• "Protocol Translation"
• "SOS Proxy"
• "Load Balancing"
• "Value Added SOS"
• "Sensor Data Store"

Georeferenced Timeseries Viewer

The Georeferenced Timeseries Viewer (GTV) is a generic desktop application and a toolbox for building specialized applications capable of presenting a common and combined view on time series data stemming from different sources, such as sensors, simulation models or data fusion outputs.

The GTV is implemented in Java on a richt client platform. It is an expert tool for the daily work of decision makers mainly in environmental authorities. The GTV is used as the basis for the ‘Data inspection client’ in Air Quality Monitoring (SP4) Pilot, and the design strongly reflects the requirements inherent to the air quality monitoring domain. Nevertheless, the GTV can be easily adopted to the needs of other environmental domains. The main design goals of GTV were:

• to develop an expert tool capable of accessing and visualization of all data used within the SANY project;
  
• to assure the GTV provides efficient and reliable support for domain
  experts inspecting large amounts of data.

• to assure the GTV is easily extendible in order to answer the future user
  requests for additional functionality

The main GTV components are a set of connectors to remote systems and a set of viewer windows, which can be combined and configured in a flexible way. Both, connectors and viewers can be easily added at runtime without the need to recompile or reconfigure the application.

SensorSA Data Acquisition System

The SensorSA Data Acquisition System (SensorSA DAS) is a network capable appliance developed by AIT, which allows seamless integration of various sensors in a Sensor Service environment based on SensorSA. The main characteristic of SensorSA is the exclusive use of OGC SWE interfaces for all communication.

SensorSA DAS exposes sensor data, management data, and history of alerts over a Sensor Observation Service (SOS) interface. The sensor configuration is performed via Sensor Planning Service (SPS) interface, and the configuration of events and alerts is performed through a Sensor Alert Service (SAS) interface.

AnySen Driver

AnySen driver has been back-ported to existing UWEDAT monitoring system and used in production environments. Companies interested in integrating AnySen in their own products are kindly asked to contact AIT.

The AnySen driver is a software component for low level data acquisition, which can be used to connect sensors to a Data Acquisition System. It controls the sensor, interprets measurement streams and parses measurements.

AnySen is a part of the Time Series Toolbox and can be easily combined with other toolbox components. The main advantage of AnySen is its capability to read and interpret data from many sensor nodes equipped with a digital interface (e.g. RS232 or LAN). This is achieved by abstracting the sensor protocols and reading the concrete description out of a simple sensor description file. All commands necessary for configuring and retrieving data from an analyser can be configured at run time. When a new sensor is attached to the DAS, the AnySen can trigger an automated configuration process, leading to the sensor level ‘Plug and Measure’.

The configuration can either be read from a central Database, or from the SensorSA Smart Sensor. The configuration possibilities include the protocol description, structure of the measurements, and various meta-information such as the name and unit of the measured phenomenon. This is a significant difference to current DAS concepts, where this meta-information is injected in higher levels, often as part of the application logic.

GeoMotes and GeoCubes

The GeoMotes are mini-GPS devices (about 30 x 40 x 10mm, antenna included) that provide an accuracy better than one centimeter, thanks to a post data processing based on a differential calculus.

These devices can be embedded in self-powered systems which are connected wirelessly to each other in order to set up a network. Each node, named GeoCube, can continuously provide GPS data to one PC. Furthermore, GeoCube is equipped with a three axis MEMS accelerometer which is able to detect high frequency displacements. Therefore GeoCube is able to send a warning, then GPS data are being recorded and new positions are computed with one hour delay.

A network of GeoCubes is a mesh with wireless links that can monitor a local area of about one square kilometer. This network is small, easy to install and a low cost solution for geohazard predictions. Each node, i.e. each Geocube, is geo-localized with a relative positioning accuracy better than one centimeter in planimetry and two centimeters in altimetry. Moreover, time accuracy of each GeoCube is better than 50ns: it can easily and precisely date or initiate events.

Microns

Microns are wireless ad-hoc sensor communication nodes developed by SDA. As opposed to the Geocube, the Micron is a sensor node and not an actual sensor. Up to eight sensors can be connected to a Micron, which in its turn stores and relays observations to the sensor network.

The Micron smart sensor nodes have an integrated battery, which allows a guaranteed autonomy of 2 years. This autonomy may vary depending of the sensors attached and the frequency of measurements. An external power source can be connected, so e.g. photovoltaic panels could also be deployed to power the sensor nodes, and take over the battery powering.

The sensor nodes support different network topologies and are organised ina self-healing network, which means that if a node fails, the other nodes will automatically find another communication route. When several routing paths are possible, they will chose the most efficient path for their data communication.

SensorSA Data Acquisition System allows seamless integration of Microns in SensorSA networks.

SensorSA Smart Sensor Adapter

The SensorSA ‘Smart Sensor Adapter (SSA)’ is a simple device which allows the user to connect a simple RS-232-based measuring device with automatic identification and registration to the station computer.

The SensorSA SSA has a possibility to provide all information required for automatic configuring of a SensorSA DAS, including e.g. capabilities of the sensor, resolution, accuracy and type of the measurements (units), sensor location, owner, proposals for information processing and more.

When attached to an USB port, the SSA allows the DAS to download its configuration data. In a next step, the DAS uses this information in the similar way it would use the configuration data obtained from the ‘sensor types’ database. Finally, the SSA switches to the ‘transparent’ mode and allows direct communication between the SensorSA DAS and the SSA RS-232 interface.

The current implementation of the SSA is based on a Microchip evaluation board shown above and the firmware supports following operations:

• setTransparent=0 or 1switch to Command mode or RS232-USB
  converter mode

• getCapabilities outputs the Sensor-ML file to USB
  
• getTemperature outputs the local temperature of the adapter (demo-value)
  
• getSwitchStates outputs the logical states of two digital input lines
  
• getPoti outputs the value of the on-board potentiometer (demo-value)

These commands can be sent over the USB connection and will be interpreted by the board until it enters the transparent mode (setTransparent=1 command). Once in transparent mode, the SSA acts as a simple USB to RS-232 bridge, which allows re-using of the existing sensor drivers with no or minimal changes.

Causal Fusion Services

Causal fusion refers to the indirect prediction of a target variable using a selection of explanatory variables. A number of causal fusion methods have been developed for use within the SANY project, including multiple linear regressions and neural networks. In most cases, historical target and explanatory variables along with real-time explanatory variables from in-situ sensors held in OGC compliant SOSs or from spatial fusion processes are accessed via an OGC compliant WPS. The resultant predictions are supplied to an OGC compliant SOS, and/or viewed through a web-interface. Two types of casual fusion algorithms were used in SANY: multi-linear regressions and neural networks.

Multiple Linear Regressions

Linear regression is used to construct a prediction formula for the target variable, given values of explanatory variables, by minimizing the sum of squared errors of linear fitting. Before constructing the linear regression formula, each explanatory variable is tested in order to determine whether a linear relationship to the target variable exists. The target variable is then predicted as a linear combination of the explanatory variables.

Linear regression is one of the most widely used modelling methods because of its effectiveness and completeness. Although the majority of processes are nonlinear in nature, many of them are well-approximated by linear models.

Linear regression estimates unknown parameters and assesses whether these parameters are statistically significant, which often has a clear meaning to scientific questions. Linear regression also assesses whether the model is statistically significant. The resulting model can be used to predict the target variable and confidence intervals.

Neural Networks

Neural networks are mathematical structures which are analogous to biological neural networks. The artificial neurons are set in layers and interconnected with each other. The neural networks are capable of processing non-linear statistical data and modelling complex relationships between inputs and outputs.

The most basic radial basis network consists of three separate layers. The input layer is the explanatory variables. The second layer is a hidden layer of high dimension. The output layer is the response of the network. The network topology is determined by the number of hidden units. One response is involved in this application.

Neural Networks are generally considered a ‘black box’ approach since the model parameters are hard to interpret in terms of physical meanings.

Spatial Fusion Services

Spatial data fusion services provide spatial trends of environmental parameters using observation data which are collated from a network of in situ sensors. This leads to the prediction of environmental parameters in areas where sensing is not available. The computation and analysis of spatial data uncertainties can also lead to identifying the areas where new sensor observations are required. Three types of spatial fusion algorithms have been developed and tested in SANY:

Krigingi

Kriging is a method of spatial interpolation, which predicts values of an environmental parameter following observations of the same parameter at a finite number of sensors locations. The spatial predictions are simply weighted averages of the observed parameter values, according to the respective distances between the sensor points respective locations. The weights in Kriging are computed so that the variance is minimised. In this sense, Kriging is often called Optimal Interpolation.

The dependency of the interpolation weights on the distances between sensors is manifested in a variogram. The Kriging variogram essentially describes the variance of the difference between two distinct spatial observations. Furthermore, a realistic modelling of the variogram, should be based on reasonably accurate observations and a good understanding of the most dominant environmental processes that influence the spatial and temporal trends of the environmental parameter under study. This is of paramount importance for good Kriging results.

The numerical procedures in Kriging additionally involve the determination of measures of uncertainty when estimating environmental parameters in a spatial domain of interest. The approach leads to a good assessment of how observation sensors should be spatially distributed for achieving minimum uncertainty in spatial fusion.

To support Kriging we have added elevation correction, periodic variable support (e.g. wind direction), automated variogram selection and multi-region variogram support.

Bayesian Maximum Entropy

Data fusion methods based upon Bayesian Maximum Entropy (BME) are able to consider soft sensor data, e.g. the sensor value lies in an interval, and additional phenomenological knowledge in the form of models. The results are statistics encompassing the uncertainty of the spatial/temporal interpolation given the uncertainty of the available information.

The overall BME fusion method is structured in three stages:

1. prior stage: consideration of general physical and scientific knowledge G about the spatio-temporal properties of the phenomenon of interest. This knowledge may, for example, be expressed in the form of spatiotemporal
   differential equations derived from physical laws or as covariance models. It is what is known before experience with the specific situation is applied. The prior probability distribution of the so-called random field of the phenomenon is determined using the maximum entropy (ME) principle, i.e. it is the most uninformative (unbiased) probability distribution given only G.

2. meta-prior stage: consideration of case-specific hard and soft data of the phenomenon of interest. This information is denoted by S (for specific knowledge) and is based on observations and measurements. Hard data refers to values believed to be accurate. Soft data is accompanied by uncertainty information such as a probability distribution for the value range.
   
3. posterior stage: processing (fusion) of the available knowledge G and S ofthe prior and meta-prior stages respectively to make a probabilistic map of the phenomenon for a given set of spatio-temporal points (typically a grid). The map is a statement of the general knowledge G relative to the case specific knowledge S and is derived using Bayesian conditional probabilities.

If the general knowledge G comprises the mean and covariance, and if S includes only hard data, then the BME estimate coincides with the simple Kriging estimate. Similarly, if G is limited to the variogram and if S includes only hard data, then the BME estimate coincides with the ordinary Kriging estimate.

When applying the BME method in the SensorSAi, the knowledge S is represented as an observation collection described with the O&M model and including uncertainty information in uncertML. The map resulting from the posterior stage
is represented as coverage with associated uncertainty information.

Socio-economic spatial correlation

This spatial correlation algorithm fuses information from an economic impact database about buildings, cracking and the potential economic impact of this cracking with ground displacement sensor data in the Barcelona region. Correlation is between each identified building and the nearest ground displacement measurement. A displacement threshold limit, determined via temporal analysis of the regions historical correlation between displacements and eventual cracking, is used to flag buildings at different likelihoods of cracking. The output is a list of buildings, spatially correlated displacement values for each building, economic information such as tax and rental income and a 'likely cracking' warning flag based on the ground displacement threshold limit.

Temporal Fusion Services

Temporal fusion can be used to predict the target variable directly from past observations of the target variable itself. The essential difference between temporal fusion and causal fusion is that temporal fusion takes the internal structure of data into account. In the SensorSA time-series data from in-situ sensors are obtained from SOS instances. The resultant predictions from the temporal fusion service are supplied to an OGC compliant SOS instance via a ‘virtual sensor’ controlled by an SPS instance.

Methods for time series analysis are often divided into two domains: timedomain and frequency domain. The frequency domain approach is more suited to exploratory analysis. The time-domain approach is discussed here. Time series usually contains some typical patterns:

• Trend: represents long-run movements in the series.
  
• Seasonal cycle: seasonality pattern repeats itself more or less around a fixed period.
  
• Autoregressive component: represents data as a function of the past history plus a white noise.
  
• Moving average component: assumes data model is a linear combination of a prior random process.

Apart from the above regular patterns, an irregular component in the time series reflects non-systematic movements in the process.

The regular patterns can be identified through exploratory analysis or empirical knowledge of the process. At this stage, one must decide the order of trend, i.e., whether it is a random walk or a local linear trend, the existence
of seasonal component and its period, the order of autoregressive and moving average components.

Two temporal fusion algorithems have been investigated in SANY: state-space modelling and Kalman filters

State-space Modelling

Once data patterns are identified, models for time series can be formed using an autoregressive integrated moving average model or state-space form.

The state-space form has enormous power to handle a wide range of time series models.

The basic structures such as trend and seasonal cycles are expressed explicitly in the model and are easy to interpret. The state-space form consists of a measurement equation and a transition equation. The transition equation contains the dynamics of the system under investigation and generates state variables. The measurement equation relates observable variables to state variables.

Kalman filters

After time series are modelled and put in state-space form, the Kalman filter algorithm may be used to produce predictions and smoothing of the statespace vector.

The Kalman filter is an important algorithm in many applications since it facilitates online estimation and enables the estimation and prediction of the state vector to be continually updated as new observations become available.

The Kalman filter is derived on the assumption that the disturbance and initial state vector are normally distributed. It gives optimal estimation of the state vector in the sense that it minimizes the mean square error within the class of linear estimators. It consists of two steps: prediction and update. The prediction step predicts the state variable and the prediction error to the next time step using the transition equation. The update step modifies the prediction once the observation at the current time step becomes available.

The Kalman filter also facilitates maximum likelihood estimation of the unknown parameters in the model. It enables the likelihood function to be calculated via prediction error decomposition. The maximum likelihood estimation can be carried out numerically or by an Expectation Maximization (EM) algorithm. The EM algorithm takes on a simple form comparing to the numerical solution and it always increases the likelihood during the iteration. The EM algorithm also tolerates missing observations and has a natural procedure to adjust the estimators.

Air Quality

Air quality is one of the most important indicators for the sustainable development. The air quality monitoring is therefore required and regulated by the law in all European states. In addition, to existing reporting obligations the
EU-wide initiatives such as INSPIRE and CAFE are gradually introducing the need for the pan-European interoperability and real time exchange of data.
The SANY ‘Air Quality’ pilot is used to validated the usability of the SensorSA based air quality management networks for three main groups of users: network operators, national environmental agencies, and for the European Environmental agency. The SANY Air Quality Management Pilot focuses on the following topics:

• Providing uniform access to data from air quality monitoring systems of France, Belgium and Austria. The Air Quality Pilot also showcases the feasibility of serving the INSPIRE-relevant meta information over the standardized OGC Sensor Observation Service interface
  
• Aiding the domain experts in performing the routine Quality Assurance of the data. This is achieved by mean of the state space fusion service. This service continuously monitors all available air quality (immission) observations and publishes the now-casts and confidence intervals at 17 measurement locations using the data model similar to the original
  immission data model. The combination of the data from both servers, presented side-by side provides a very effective help in finding suspicious measurements.

• Identifying the impact of the known pollution sources to actually measured immission, and providing an indication for the relative importance of the unknown (unaccounted for) sources of pollution at the selected positions. This is achieved by comparing the immission measurements with the prediction based on real-time emissions from major industrial plants in the Linz area.
  
• Illustrating the feasibility of the automatic report generation. This use case is limited to automatic generation of the data required for reporting in the CAFE

Marine risk

SP5 prototype can be explored at SP5 application web page. In case the demonstration is currently offline, please contact BMT.

Marine risks arise from a number of sources including natural events, anthropogenic causes and a combination of both. In almost all cases, marine risks have an economic, human and environmental impact. In SANY, short term microbial contamination of both Bathing Waters and Shellfish Waters has been targeted. These designated water areas are subject to extensive regulatory standards, established via EU Directives.

In the case of Bathing Waters, failure to meet regulatory standards can have significant impacts on public health and tourism revenue. Similarly, short term microbial pollution events in Shellfish Waters can have serious consequences for consumer health and cause a reduction in revenue for the local aquaculture industry. Presently, microbial levels in the selected water areas are assessed using laboratory testing. These tests often have a turnaround time of more than 24
hours and, as such, can only determine whether a contamination event has occurred. Improvement in the ability to forecast the risk of short term microbial pollution in designated waters could reduce both the human and economic
impact of such events. The SANY marine risk applications use a number of services developed within the SANY project to access data from sensor networks and assess the likelihood of a contamination event occurring. The use of SANY
Sensor Service Architecture enables:

• Access to third-party sensor networks and phenomenological models, to create cost-effective access to measured or modelled data streams and equally to allow operators of such networks/models to valorise their investments;
  
• The use of web-based services to provide high-value data processing (eg for spatial fusion, temporal fusion and modelling) that will enable users to get enhanced information about parameters of interest;
  
• Rapid deployment of additional in-situ sensors on both fixed and mobile platforms. These will acquire data on key water quality parameters, to fill gaps in spatial and temporal data coverage, and thereby permit improved quality of risk now-casting and forecasting;
  
• Provision of alerts and alarm systems to raise the awareness on a possible hazard and support preventative measures;
  
• Remote configuration of smart sensors and, if possible, adaptive tuning of stochastic models to allow ‘on the fly’ enhancement of risk forecasting through incorporation of recent data within the forecasting algorithm.

Urban Geo Hazards

Geo-hazards may be caused by human activities or natural events. Whether those hazards are induced by human activity or natural hazards, they have an economic, human and environmental impact, which cannot be neglected. As an example, landslides are among the most widespread hazards on Earth causing billions of dollars in damage and thousands of deaths and injuries each year around the world, and Europe has the second highest incidence of landslide casualties of any other continent. As well, recent accidents in European cities induced by construction works raised the awareness for a better control of monitoring data, and enhanced services for decision support.

In SANY, the geo-hazards pilot focuses on hazards related to construction works in dense urban areas. Indeed, with the expansion of urban areas and the densification of population and transport networks of those areas, construction and rehabilitation works on structures have become more frequent, thus the population is exposed to higher risks. There is therefore a critical need for a better management of geotechnical risk in such a context.

Moreover, monitoring systems and sensor management software installed on a construction site are usually proprietary, and vary from one provider to the other, thus multiplying data sources and information. With respect to those limitations, the Geo-hazard application intends to provide an easy and fast access to sensor data, independently from the sources, and the possibility to merge that information through fusion and modelling services, in order to offer synthetic and comprehensive information to the end-user.