The SQL-Based Geospatial Web Processing Service
The SQL-Based Geospatial Web Processing Service
net/publication/229041794
CITATION READS
1 440
2 authors:
All content following this page was uploaded by Soravis Supavetch on 10 March 2019.
Abstract: Since the OGC promoted the specification of OGC Even though WPS fulfils the gap of transition from old
Web Processing Service (WPS), several researches on the traditional standalone GIS through service-based architecture,
development of geospatial online service have been happened. no real geo-processing capabilities do exist. For reasoning of
Most of WPSs are developed extending on GIS software or APIs.
ignoring re-implementation an existing GIS function, [1]
Nowadays, most databases are extended to support spatial data
and also provide a number of spatial functions which can be developed WPS using GRASS GIS for a processing engine
called through SQL. This paper presents the development of and open shell script through WPS interface to user for
WPS which uses spatial database as processing engine. Extended executing GRASS functions. On our perspective, not only GIS
WPS protocol is designed to support SQL to invoke spatial software can be used as WPS back-end processing engine, but
function. Using SQL eases the modification of WPS regarding also spatial database is suitable. Spatial database nowadays
the change of processing function service. Databases tested with
provides numerous spatial functions. In addition, the spatial
the implemented WPS include Oracle, PostgreSQL and MySQL.
The database type is listed in WPS GetCapabilites document and functions can be customized or extended through a stored
allowed to choose for working spatial engine. Thus, users can use procedure.
SQL in which they are familiar with the database. The results Currently, several relational databases provide spatial
from scenario tests clearly show the success of the engine to perform several spatial operations which can be used
implementation of this concept. with stored data through SQL without the need of external GIS
tools. If spatial database can be used as the spatial engine for
Keywords: Web Processing Service, Geospatial Processing
Service, Geospatial Processing Engine, Spatial Database. the back-end processing of WPS, this will help reduce
workloads for programmer to develop spatial operation tools.
Moreover, if SQL statement can be attached in the request
I. Introduction
form, the translation of XML in the request form invoking the
Traditional GIS systems are no longer suitable for modern GIS functions can be dismissed since it is directly sent to
distributed, heterogeneous network environments due to the spatial database for carrying out the GIS function. It is also
lack of interoperability, reusability, and flexibility [2]. The flexible for users to form SQL with knowledge on using
geographic information service is enabled by the advancement back-end spatial database used in WPS.
in general web service technology and the efforts of the GIS This paper reports the implementation of WPS using the
industries. Due to the popular use of the Internet and the spatial database as the back-end spatial engine for processing
dramatic progress of telecommunication technology, the wfs:FeatureCollections data from open WFS site. This
paradigm of GIS is shifting into a new direction [6], especially research is an adaptation of our previous research [3] in which
for producing spatial-oriented visualization [5] in personal WPS request form is extended to support the additional basic
devices such as cell phones, PDAs, and smart phones [4]. Until SQL (SELECT-FROM-WHERE-FILTER) for calling the
the Open Geospatial Consortium (OGC), the non-profit, processing task from available databases in WPS. Oracle
international, voluntary consensus standards organization is Spatial, PostGIS and MySQL are databases that are used in
leading the development of standards for geospatial and this research.
location based services. Several implementation Next section gives a short insight into WFS, WPS and
specifications have been deployed, and some of them are Spatial Database. The detail of extended spatial SQL for WPS
widely accepted in GIS industries e.g. GML, WMS, WFS, and request is presented. The developed WPS architecture and
WPS. These services open the opportunity to solve the back-end procedural composition are introduced. The
problems that are typically faced in traditional GIS. The OGC processing capabilities of developed system are described.
WFS [8] defines interfaces for data access and manipulation The test of implemented service through some scenarios is
operations on geographic features. The OGC WMS [10] demonstrated.
specifies the behavior of a service that produces image maps.
The OGC WPS [11] defines a mechanism by which a client II. Technologies Reviews
may submit a processing task to server. To implement these
OWSs supporting scientific workflow require services A. WFS (Web Feature Service)
composition called “web services chaining” [7], “web services
composition”, “web services flow” or “web services WFS enables a client to retrieve geospatial data encoded in
orchestration” [12]. Geography Markup Language (GML) online through HTTP
protocol. It allows for clients and servers of different vendors A. Processing Clause
or systems to share data without having to convert data Only SELECT clause from database SQL is un-extended. User
between proprietary formats. WFS server is then a feature can use spatial functions e.g., SDO_CS, SDO_GEOM,
online resource, providing an interface for data access and SDO_LRS or SDO_UTIL package subprograms from Oracle,
manipulation. A request sent to a WFS server is a query (or ST_Buffer, ST_DWithin, ST_Transform,
transformation operation) for one or more features that can be ST_Line_Interpolate_Point functions from PostGIS, or
delivered by a server. When the WFS receives the request, it functions from MySQL, to process features from WFS site.
executes the function by the request and sends back the Another advantage of using spatial database as processing
information to a client. WFS communicates with a client and engine is that user can customize additional function
allow programs written in different languages and on different embedded to database through stored procedure. More
platforms. Interface that is typically used in WFS is defined in developed algorithms can be plugged into service. The
XML. processing function in SELECT clause is not limited in one
function per request but user can request numerous examples.
B. WPS (Web Processing Service) For example SELECT ST_Buffer( ST_Transform(…) , 100 ),
ST_Buffer( ST_Transform(…) , 200) to retrieve two different
buffered polygons from single original polygon within
WPS is designed to standardize the way that GIS calculations
coordinates already transformed.
are made available to the Internet. WPS can describe any
calculation (i.e. process) including all of its inputs and outputs,
and triggers its execution as a Web service. WPS supports B. Data Retrieving Clause
simultaneous exposure of processes via HTTP/GET (KVP:
Key Value Pairs) and HTTP/POST (XML-based document), This processing service provides only processing functions
thus allowing the client to choose the most appropriate without any provided features. Processing features must
interface mechanism. The specific processes served up by a retrieve from WFS site. We extended an SQL in FROM clause
WPS implementation are defined by the owner of that to express WFS site and attaching FILTER clause for defining
implementation. filter condition. The address of WFS site is located in the
FROM clause. The layer name is defined after the address
C. Spatial Database separated by #. The address and the layer name are both
encoded together inside <?--?> tag. For filtering condition,
Spatial database is a database system which offers spatial data user can express OGC Filter Encoding like SQL. We use CQL
types in its data model and query language. Most relational filter (Common Query Language) from GeoTools
database management systems have supported spatial data (http://www.geotools.org) to wrap FE (Filter Encoding) [9]
type including point, line and polygon. Some of them include expression. The following table is examples of CQL filter
Oracle, MySQL and PostgreSQL. MySQL has included comparing with OGC FE syntax.
spatial extension built in since MySQL 4.1 which supports
data type following OGC simple feature specifications for CQL Filter Encoding
SQL, and also provided spatial query tools and supports L.NAME = ‘LAND01’ <PropertyIsEqualsTo/>
spatial indexing using R-tree with quadratic splitting. PostGIS
L.NAME <> ‘LAND01’ <PropertyIsNotEqualsTo/>
is used to spatially enable the PostgreSQL to be used as a
backend spatial database. PostGIS uses an R-tree index L.AREA > 100 <PropertyIsGreaterThan/>
implemented on top of GiST (Generalized Search Trees) to BBOX(GEOM,
<BBOX />
index GIS data. Oracle Spatial is a module option for Oracle 10,20,30,40)
that provides advanced spatial features to support GIS CONTAINS(GEOM, … ) <Contains/>
solutions. OGC simple feature is supported since Oracle 10g.
INTERSECT(GEOM, … ) <Intersect/>
An R-tree or quad tree index can be used in Oracle for spatial
data index. Table 1. A Comparison of CQL Filter and OGC Filter
Encoding
III. Extend Spatial SQL for WPS
Syntax Rules
SQL as processing command is the main idea of this research
to explore processing capabilities of spatial database to INSERT, UPDATE and DELETE clauses are not allowed
web-based user. An SQL base syntax for this service. User only use SELECT clause to request a
SELECT-FROM-WHERE clause is applied to processing function. The structure of extended SQL statement
SELECT-FROM-WHERE-FILTER for WPS user request. is as follows:
Extended SQL is designed for user to send a request statement
to WPS which enable user to define a GetFeature from WFS SELECT <Processing Function>
site with processing simultaneously. Processing functions are FROM <WFS Site>
available in our WPS coming from spatial database. The WHERE <Processing Condition>
pattern of designed SQL statement is shown in the following FILTER <CQL Filter>
section.
An expression in the SELECT clause is a native SQL
The SQL-Based Geospatial Web Processing Service 121
SELECT L.NAME,
MBROverlaps(L.SHAPE,’determine’,
Processing Invoker Processing engine
GeomFromText(‘POLYGON((
Engine
101.86 14.81,101.86 14.92,101.97 14.92
,101.86 14.81))’),0.1) AS RELATIONSHIP Response
FROM <?http://localhost/geoserver/wfs#nesdb:LAND?> Data Transformer
GML / JSON
FILTER NAME = ‘LAND01’
Figure 1. Online processing service frame work.
The first example is to evaluate the polygon relationship
between the land parcel ‘Land01’ which is queried from When the user send the request via HTTP GET or POST
another WFS and the defined polygon. Such processing with KVP encoding which is a designed statement including
occurrs in Oracle database. The second example is the same SQL for processing function, SQL parser will interpret and
question as the first example, but the spatial engine used is extract a WFS data source and SQL statement from the user
MySQL database. request to form a WFS request. WFS requester will send such
request to WFS site to get data in GML format. Once GML
stream is responded back from WFS site, the data importer
IV. A WPS ARCHETECTURE will transfer that received data into the database store. After
that, the processing invoker will send the SQL command to the
The developed WPS used existing spatial database, Oracle spatial database to call spatial processing functions. The result
Spatial, MySQL and PostgreSQL, as a spatial engine. Several set from spatial database will be parsed to a data transformer
tools were developed for many purposes, e.g., to support for converting result set to user defined output format: GML or
extended SQL request form, to connect and retrieve data from JSON (JavaScript Object Notation).
other WFS sites and to manage and manipulate collected GML The sequence diagram of above processing tasks is
data. The architecture of the service system is shown in demonstrated as follows:
Figure 1.
122 Supavetch and Chunithipaisan
Feature Type
Success
Metadata
Processing
Create Spatial Metadata,
Features data Create Temp Table
GetFeature
Figure 2. Online query and processing service (OQPS)
sequence diagram Features Data
A. SQL Parser
WKT Features
An SQL Parser component is the most importance part of the Import Features
service. Every incoming SQL composes of two conditions i.e.,
data retrieving condition and data processing condition. Data Import Success
requests from FORM and FILTER clause will be re-encoded
Features ready
in WFS request protocol in order to parse to WFS data source. for processing
Data that are retrieved from WFS site will be stored in ad hoc
database table. The user requests statement is then reformed
into database native SQL for sending to user defined database Figure 4. Preparation diagram for temporal feature
to invoke spatial operation. importation.
Requested SQL
C. Data Importer
SELECT <Processing Functions>
FROM <URL#TypeName>
WHERE <Processing Condition> Data Importer imports data from WFS requester into prepared
FILTER <CQL Filter> table for ad-hoc dataset to be used in the processing task later.
Data Importer transforms GML feature members into SQL
insert statement and performs execution. For using Oracle
database, after data inserted into table, Data Importer will
Executing SQL
invoke creating spatial index statement (Oracle also requires
SELECT <Processing Functions> spatial index for performing spatial query and processing).
FROM <Temporary table>
WHERE <Processing Condition> D. Processing Invoker
User can use any SQL statement in which database that the
user is familiar with. The detail of using the SQL_Processing
service is found at DescribeProcess operation. It describes the
parameters that have to be sent to the service, the type and
value of parameters to be defined, and the output type of
response value. The example of DescribeProcess document of
Figure 5. GML result.
SQL_Processing is shown as follows:
From the above example, RD_LENGTH and THE_GEOM
<DataInputs>
are used as alias names of feature attributes with the attribute
<Input minOccurs=“1” maxOccurs=“1”>
data coming from the processing operations, whereas,
<cuwps:Identifier>SQL</cuwps:Identifier>
RDLNNAMT is the attribute which already come from the
<cuwps:Title>SQL statement</cuwps:Title>
requested WFS and can be directly put in GML. <cuwps:Abstract>SQL statement
Another format, JSON, is also supported in the Data </cuwps:Abstract>
Transformer tool. This type of output can be directly used in <LiteralData>
client side application which JavaScript is supported, e.g. <cuwps:DataType ows:reference=”xs:string”/>
OpenLayers and GeoExt. JSON data is encoded in the format <cuwps:AllowedValues>
so-called JavaScript array object which is used to store <cuwps:Value/>
multiple values in a single variable. Some JavaScript library </cuwps:AllowedValues>
can benefits from JSON format for data visualization by </LiteralData>
rendering data in table form, for example, Ext.data.ArrayStore </Input>
class in Ext. The following example shows the output in JSON <Input minOccurs=“1” maxOccurs=“1”>
format from the same user request statement. <cuwps:Identifier>DATABASE</cuwps:Identifier>
<cuwps:Title>Database name</cuwps:Title>
SELECT SDO_GEOM.SDO_LENGTH(…) <cuwps:Abstract>Database name
AS RD_LENGTH, </cuwps:Abstract>
RDLNNAMT, <LiteralData>
SDO_UTIL.TO_WKTGEOMETRY( <cuwps:DataType ows:reference=”xs:string”/>
SDO_AGGR_CONCAT_LINES(…) ) <cuwps:AllowedValues>
AS THE_GEOM <cuwps:Value>ORACLE_10G</cuwps:Value>
FROM <?...?> <cuwps:Value>MYSQL_5</cuwps:Value>
FILTER … <cuwps:Value>POSTGIS_8_4</cuwps:Value>
</cuwps:AllowedValues>
</LiteralData>
</Input>
[<string: RD_LENGTH>,<string:RDLNNAMT>,<string:WKT>] </DataInputs>
<ProcessOutputs>
Figure 6. Additional JSON output format. <Output>
<cuwps:Identifier>GML2</cuwps:Identifier>
124 Supavetch and Chunithipaisan
Acknowledgments
Author Biographies
This research was supported by the Ratchadaphisek Somphot Soravis Supavetch received his B.Eng degree in Survey Engineering from
Endownment Grants from Chulalongkorn University. Chulachomklao Royal Military Academy and M.S. (Forestry) degree from
Kasetsart University, Thailand. He is currently a PhD candidate in Survey
Engineering at Chulalongkorn University, Thailand and currently attached to
References Royal Thai Survey Department. His area of interest includes service oriented
architecture and geospatial processing service in geospatial information
system.
[1] Brauner, J., and Schaeffer, B. “Integration of GRASS
Functionality in Web based SDI Service Chains.” Free Sanphet Chunithipaisan received his B.Eng and M.Eng degree in Survey
and Open Source Software for Geospatial conference, Engineering from Chulalongkorn University, Thailand. He holds a PhD in
Geomatics from University of Newcastle upon Tyne, UK. He is currently
Barcelona, Spain, 2008. working as Assistant Professor in Department of Survey Engineering at
[2] Chen, L., Xiujun, M., Guanhua, C., Yanfeng, S., and Chulalongkorn University. His area of interest includes spatial database and
Xuebing, F. "A Peer-to-Peer Architecture for Dynamic data interoperability in geospatial information system.