Proxy4OWS/source/owsproxy.rst

Proxy4OWS
#########

*Proxy4OWS [http://redmine.ccss.cz/projects/owsproxy] is Open Source server
program, which enables to visualize and work with large vector data and
raster data in the web environment. It basically transforms OGC Web Feature
Service and OGC Web Coverage service to OGC Web Mapping service calls. It
can be also used for transformation of the original data into
server-unsupported coordinate reference system, including OGC WMS*

The problem of displaying large GIS datasets
============================================
When working with large vector datasets, we are usually facing limits of
nowadays browsers. Google Maps [#]_ for example are limiting number of
displaying vector features to 1000. In OpenLayers [#]_, no limit for number
of features is used, but displaying e.g. cadastral map makes browsers often
freeze. Advantage of working with vector data directly is (among others),
direct possibility of vector data editing, more interactive way of user
experience, speed (when dealing with reasonable amount of data).


While vector data can be displayed using todays technologies, some popular
raster data formats, such as GeoTIFF, can be not. According to W3C [#]_,
only few formats for raster data are supported, none of them is really used
or usable in GIS, usually because of compression method, they are using. In
the internet, raster graphics format are focusing on possibly low amount of
data transfered from the server to the client, while loosing the accuracy
of the data being transfered. In GIS, we are focusing on data quality,
regardless on file size.

Raster and vector data are usually distributed using OGC OWS standards.
Vector and raster data are distributed via OGC Web Feature and Raster
Service respectively. Both services are offering list of datasets and
metadata.

Another problem might occur, when some OGC Web Mapping Service does not
offer coordinate reference system, in which the web mapping application is
configured. Some middle-ware have to be setuped between the map application
and the server, which will transform the images from server coordinate
reference system to the one, accepted by the client.


Proxy4OWS 
=========
Proxy4OWS is server script, which is between the client mapping application
and OGC OWS server (WCS, WFS or WMS). It is transforming OGC WMS request
types from the client, into WCS or WFS requests, so that the target server
can accept them. On the way back, it downloads the data distributed by the
server (raster or vector), creates image and sends it back to the client.

Features
--------
It also transforms GetCapabilities request-response pair, so that the
(WMS) client can deal with it.

As result, data are processed on the server, into the form, common
web browser mapping application are able to display without big demands on
system resources.

.. figure:: imgs/owsproxy_sequence_diagram.png
    
    HSlayers.Layer.WCS and WFS are derived from Layer WMS. As proxy
    between the client (the map) and the WFS|WCS server, Proxy4OWS is
    placed. Proxy4OWS transforms WMS requests comming from the client
    into WFS|WCS calls and also responses are transformed to WMS
    responses or images, displayable in the web browser.

Proxy4OWS also can deal with OGC WMS service in the way, that it is
transforming the coordinate reference system of the original service into
the client-preferred one in the case, server does not support coordinate
system of the client. The resulting image is usually a bit distorted, yet
displayable. 


Technology
----------
Proxy4OWS is written in Python programming language. Following libraries are
used:

MapServer [#]_
    MapServer is the core of Proxy4OWS. Proxy4OWS generates the Map object
    configration and after that dispatch method of MapServer is used, which
    will deal with the request, download all necessary data from servers
    and generate resulting image.

    From the client-point of view, it is used for working with vector data
    directlya (deals as OGC WFS client).
    
GDAL/OGR [#]_
    GDAL is used for reading OGC WFS and OGC WCS service metadata, so that
    the WMS response from Proxy4OWS to the client, has all necessary
    informations.

    While for OGC WFS service, MapServer directly is acting as client, OGC
    WCS is configured as GDAL data source.

    Also for WMS transformation service, WMS interface of GDAL is used,
    because of, it is able to deal with tiled requests, preserving the
    large dataset exceptions issue.

OWSLib [#]_
    OWSLib is Python library, which acts as OWS client. With help of
    OWSLib, metadata of particular target services are being collected
    easy.

Architecture
------------
Once again: Proxy4OWS acts as WMS server to the client and acts as WFS, WCS
or even WMS client to the target server.

GetCapabilities
    When GetCapabilities request arrives from the client, Proxy4OWS
    Checks for existing cached directory with mapfile, creates new, if not existing
    
GetMap
    When GetMap request arrives, image is generated based on previously
    generated mapfile, using `OWSDispatch` method. At this point, WFS
    filter is applied, if target server is WFS.

In both cases, Proxy4OWS is looking for existing MapServer MapFile (it
creates one, if it does not exist) and let MapServer do the work. Proxy4OWS
is takes care about proper MapFile configuration.

.. figure:: imgs/getcapabilities.png
    
    When WMS GetCapabilities request arrives, Proxy4OWS generates MapFile
    with list of layers, corresponding to either feature types (WFS) or
    coverages (WCS) of the target service. For configuring the MapFile
    properly, GDAL, OGR and OWSLib libraries are used. 

    WFS Layers are configured, using MapServer as WFS Client, while WCS
    layers are using GDAL as WCS Client.

    Then MapFile is generated, `OWSDispatch` method is called and
    Capabilities response does arrive.
    
.. figure:: imgs/getmap.png

    When WMS GetMap request arrives, Proxy4OWS find existing MapFile
    (creates new, if it does not exist) and performs `OWSDispatch` function
    of MapServer, which generates the output image and sends it back to the
    server.

Invocation
----------
Proxy4OWS is originally designed as WMS server. But it can also be used as
WFS or WCS server, so that it can only transform original data into
coordinate system unsupported by the target server.

Therefore, the client can use basically any type of OWS request using
proper parameters. In addition to this, two more parameters will have to be
appended to the request URL:

    * `owsservice` says, what is the target server service type (WCS, WFS
      or WMS)
    * `owsurl` is the URL of original server

.. figure:: imgs/crwfs.png

    Image based on OGC WFS service, as result of following WMS GetMap
    request. Note `owsurl` and `owsservice` parameters in the URL:

    `http://localhost/cgi-bin/owsproxy.cgi?map=/var/www/localhost/htdocs/tmp/WFS-8066ca5a58f48c499f919d45529596c0xeWyOF/mapfile.map&owsService=WFS&owsUrl=http://bnhelp.cz/ows/crwfs&LAYERS=kraje&TRANSPARENT=TRUE&FORMAT=image/png&EXCEPTIONS=XML&VERSION=1.3.0&INFO_FORMAT=application/vnd.ogc.gml&CRS=EPSG:3035&SERVICE=WMS&REQUEST=GetMap&STYLES=&BBOX=2844318.7336584,4288032.8774186,3211207.4244274,5053560.2418116&WIDTH=1085&HEIGHT=520 <http://localhost/cgi-bin/owsproxy.cgi?map=/var/www/localhost/htdocs/tmp/WFS-8066ca5a58f48c499f919d45529596c0xeWyOF/mapfile.map&owsService=WFS&owsUrl=http://bnhelp.cz/ows/crwfs&LAYERS=kraje&TRANSPARENT=TRUE&FORMAT=image/png&EXCEPTIONS=XML&VERSION=1.3.0&INFO_FORMAT=application/vnd.ogc.gml&CRS=EPSG:3035&SERVICE=WMS&REQUEST=GetMap&STYLES=&BBOX=2844318.7336584,4288032.8774186,3211207.4244274,5053560.2418116&WIDTH=1085&HEIGHT=520>`_


WMS Transformation
------------------
Implementation of transformation of OGC WMS services is done in little bit
different way. Since it is assumed, that Capabilities document is already
parsed, it is expecting GetMap request from the client to Proxy4OWS
directly. The GetMap request is expected to have - next to original WMS paramters -  also
three add-on options:

    * owsService - this is going to be WMS
    * owsUrl - URL of the original service, which is expecting to handle
      the GetMap request
    * fromCRS - CRS of the original coordinate system, from which shall the
      result of GetMap be transformed to.

The MapServer's mapfile is generated on-the-fly. Only one layer is attached
to the mapfile - layer of type WMS. MapServer then formulates the necessary
request, fetches the data from remote server and provides image
transformation on them. Result is always little bit distorted, because the
resolution is not always fine enough, but the it can be used and displayed
in the mapping application.

.. figure:: imgs/wms_sequence.png

   WMS Sequence diagram. 
    

.. figure:: imgs/wms_transformace.png

   WMS transformation result - left map coordinate system, right -
   transformed result from EPSG:4326 source.

Thanks to Proxy4OWS, we can now display seam-less data from several WMS
resources, which do not support coordinate system of the map, displayed in
user's browser.

Further development
===================
Currently, when full featured Capabilities response is send back from the
server to the client, for some servers, with hight amount of data, this
takes long time. 

For WFS for example, GetCapabilities, DescribeFeatureType and sometimes
even GetFeature requests are to be called, before all necessary metadata
are filled to MapFile. This certainly makes the Capabilities response come
back after long time. In the future, we would like to eliminate this,
calls, so that only smallest amount of requests (GetCapabilities) would
have to be requested, and WMS Capabilities response could be returned
possibly fast. Of course, when users chooses the particular layer to be
displayed, additional non-standard call would have to be done, for getting
addition attributes, so that the client has all necessary informations
(which can be normally taken from Capabilities document).

Also now, virtually no configuration caching is created. For each
GetCapabilities request, new temporary MapFile configuration is setuped.
Proper way would be to use already existing MapFile for particular service,
if already generated and evaluate the difference between cashed version and
potentially upgraded service.

Proxy4OWS is not cashing any data. It would be the option for further
investigation, if it would make sense, to pre-cache some original data at
Proxy4OWS side, and make the performance higher.

.. [#] http://code.google.com/intl/cs-CZ/apis/kml/documentation/mapsSupport.html
.. [#] http://openlayers.org
.. [#] http://www.w3.org/Graphics/
.. [#] http://mapserver.org
.. [#] http://gdal.org
.. [#] http://sourceforge.net/apps/trac/owslib/wiki