Thursday, January 6, 2011

No Cost GIS Mapping Packages for Natural Resources Data

Overview

The sticker price for some of the high-end commercial mapping software can be a concern with regard to widespread deployment of mapping capability. However, there are quite a few free GIS mapping packages available, some surprising in their capabilities, and a number of these offerings are explored in this article. While the mapping applications may be free there are trade-offs involved in their usage in a corporate environment, some technical and some political. Further, there are sound reasons for avoiding the route of free software completely, such as continuity, user and technical support and security. Despite the drawbacks of free and/or opensource mapping software there are times when having a capable package available, ready, tested, approved and ready to go could be a critical factor in a number of situations.

For most of the larger oil and gas upstream or midstream companies the possibility of adding an opensource or free mapping package to the mix of supported software packages is a non-starter. However, there are a number of scenarios where use of a free mapping package could serve a critical role in a corporate environment, such as; emergency deployment to field personnel in advance of a hurricane, unexpected data room excursions or dispersed field work with a large number of personnel. Further, new entrants to the natural resources arena, companies which currently have no mapping capability or companies or consultants which have mapping packages foreign to that of an operating partner may be candidates for a usable and robust third-party mapping package, which in this case would be a no-cost solution.
Over 20 GIS/mapping packages were evaluated for this review. Links to most of the packages which survived the first pass are presented as hyperlinks. It is possible to use the linked information as the basis for your own or for an internal company review of the free GIS mapping offerings available in Spring 2010.

Internet Resources for Free and/or Opensource GIS Mapping

Links to free, open source mapping applications were found by various means, mainly Google, plus the following resources:

OSGEO – The Open Source Geospatial Foundation supports “collaborative development of community-led projects.” The site serves up a number of OSGeo Projects for Web Mapping, Desktop Applications, Geospatial Libraries and a Metadata Catalog.
Open Source GIS lists a number of applications and utilities relating to work with maps. Althgouth currently, the page indicates that the site has not been updated since 2008.
FreeGIS lists a number of free datasets, projects and applications relating to mapping.

SourceForge is a site maintaining a large number of open source projects, including mapping apps.

In some cases the actual home of an application is unclear. Several packages, GRASS, in particular, have moved relatively frequently. As packages lose general support or interest from their development team they are likely to fall into stale links.

Utilities

A number of utility packages may provide useful functionality; however, these are not standalone applications meeting the criteria of this evaluation. Some of these sites will only be of interest to developers.

GMT is a command-line mapping package
FWTools has a number of utilities and pointers to other Open Source utilities.
GeoTools, Java GIS toolkit
MapNik, toolkit for cartographic enhancements
JTS Topology Suite, Topology tools, also ported to C++ and used in some of the evaluated apps
GDAL geospatial translator for data formats, also maintains the OGR tools
Colorbrewer for selection of appropriate colors in thematic maps.
SpatialReference EPSG codes and reference for coordinate systems

Evaluation Scenarios

Several criteria were developed to evaluate GIS mapping applications. Since there were a large number of packages available, criteria were created to reduce the population of evaluated applications to a handful.

The most important elements of selection are based on ease-of-use and installation. Further, the application interface must present the user with a Graphic User Interface (GUI) and run on a Windows machine. Multiplatform operation is a bonus, but Windows was considered essential as this is the dominant OS for corporate deployment. From a technical standpoint it was a requirement for the mapping package to read shapefiles as well as TIFF files. Further the data should be read and not translated into another format for consumption by the package. The application is assumed to run as a standalone without need for internet connection for licensing or data. Ideally, the application should install from one package and not need the internet for additional components.

For evaluation purposes the mapping program is intended to act mainly as a viewer with limited analytical capabilities. The reason for this restriction is simple, analytical operations between different packages would require a significant amount of testing to corroborate relative to one another as well as to a corporate standard, such as ArcGIS. Handling and testing of multiple geoprocessing tasks in projected versus geographic space were deemed beyond the scope of this brief review. However, if a package had the capability of some of the more common geoprocessing operations they were noted.

Within the context of the Evaluation Scenarios it was assumed that persons with little or no mapping background or whose main area of expertise is in an area other than that of mapping or cartography would be running the program. Therefore, a program with a simple, intuitive, well-designed interface is desirable. Further, the application should have a small footprint as the map program and related data may not represent the prime area of interest of the user. For this reason, GRASS was excluded from review due to its complex nature and programmatic footprint of over 280Mb.

The addition of a free GIS mapping package is assumed to not displace existing mapping applications within a corporate environment. The fact that a free or no cost mapping application may not be as robust as a commercial product is a given. Further, these applications may not handle complex geometries or perform as a well tested commercial application with a long history. The ability of the free mapping application to handle errors gracefully is important.

Licensing Issues

For use by a third-party or employee in the field, the licensing of the GIS/Mapping package must allow for its use in a commercial setting. Most applications reviewed have either the GPL or Lesser GPL license, meaning that the packages are free to use, the Lesser GPL license indicates that components of the source may be used in a commercial application. Others, such as ArcExplorer and ArcGIS Explorer are licensed similar to other ESRI commercial products, but are still free to use. LandSerf does not declare a specific type of license, however, it does restrict copies of the package for personal or education use only, therefore this restriction was assumed to exclude commercial use, thereby eliminating the package from consideration.
Many packages provide source code, and some allow for external development against the application framework. In other words, customization is possible in nearly all applications. In the case of modification of the source it may be necessary to review the license to determine if the modifications must be sent back to the copyright holder.

Installation

Ease of installation was considered critical as outside of an emergency this type of application is unlikely to receive support from an internal IT group. In most cases, prompt support for this kind of third-party application would be a user group in support of the application, the application’s authors or if the application was presented by a corporation to a third-party for use on their behalf, then, probably, a corporate GIS Group would take the lead in support of the application. Therefore, the ability for a field person or consultant to install a mapping application with a minimum of technical difficulty is essential as these persons may be remote and/or distant from technical support as even the best and most expensive commercial applications of any kind sometimes have problems with installation.

Mapping applications lacking an .EXE or .MSI for automatic installation were not further considered, with one exception. This requirement eliminated what may be some promising, mainly Java, packages from evaluation, such as: KOSMO, SAMT, geOxygene and OpenMap. Also, for the evaluation, the install and application environment must be presented to the user in English. This requirement eliminated savGIS (French) and hidroSIG (Spanish) consideration.

Other install mechanisms were considered such as Zip and RAR files. These can sometimes present problems when a decompression program is not available. Windows now handles Zip files, however, RAR files need an external program, such as WinRAR. To simplify testing and evaluation, applications distributed as RAR files were excluded from consideration. For example, the beta version of Diva-GIS v7 was excluded, as it was only available as a RAR file, however, the current stable version, 6.0.2, was available as a Windows executable install. Zip files which unzipped to an executable were allowed in for evaluation. ESRI’s ArcExplorer v9.3.1 is such an install.

Programmatic Currency and Stability

For the evaluation the application environment must have shown both recent and ongoing development. Opensource packages presented on Sourceforge, a software project site, with build dates or installer executables of over two years old were not further considered. This requirement eliminated Fmaps and OpenEV from consideration. Software and operating systems change constantly and it seems likely that a package reflective of ongoing work in enhancements and chasing bugs would demonstrate a more stable working environment and greater likelihood to be in harmony with newer operating systems and/or common language runtime environments. It is a theory. However, with older machines or operating systems, then an older mapping application may be perfectly appropriate, and, in fact, may be the best choice.
On the other hand, free and opensource mapping applications are often presented in alpha or beta states to allow for expanded testing and debugging during development prior to deployment of a stable version. These types of packages were not deemed suitable for the evaluation. This requirement eliminated iGeoDesktop and SharpMap from consideration.

Technical Requirements

For the Base Case, at minimum, the GIS application must support the addition of ad hoc shapefile and TIFF images. This simple requirement excludes ArcReader which might otherwise be an excellent deployment vehicle for external map deployment from ESRI shops with ArcPublisher. The evaluation scenario assumes the client machine is working standalone and datasources existing on the internet would not be available. In other words, packages which rely strictly on data from WMS (Web Mapping Service) or other network mapping protocols were not considered.

Typical workflows in the natural resources sector involve numerous map layers. The Extended Case evaluation uses file sizes typical in oil and gas workflow for testing, which is significantly larger than that of a casual user. With a large number of layers the ability to save or preserve work done in preparing the layers for display is essential. In this regard, the concept of a project file to persist the state of the program and/or program settings is important for ease of use in order that the time-consuming work of symbolizing various layers need only be done once.

Base Case

All Base Case evaluations were made on a 3Gb, 3Ghz Dual Core machine running XP. Ideally the tests would be run on a ‘clean machine’ for each install; however, this was not possible. The workflow for the Base Case evaluation of the software packages was simple. A USGS shapefile polygon of the grid of 7.5 minute degree quadrangles for Texas was used as the vector layer. The raster layer used was one of the Digital Raster Graphic (DRG) TIFF images of the Katy Quadrangle for Texas. Both map elements were in a geographic coordinate system (GCS), with a NAD83 datum.

DRG rasters have a fixed color palette of 13 colors which enhances compressibility and mimics the seven color spectrum of the original USGS quadrangle map in hardcopy. On occasion DRG files may be misread by common TIFF viewers. However, failure to read a USGS DRG is probably not a fatal flaw for use of a mapping application employed in non-US locations.
Several packages were unable to process TIFF files for DRG or other flavors of TIFF; notably SAGA and Thuban errored on adding TIFF files to the map. DIVA-GIS failed to recognize the USGS DRG as being north of the equator. gvSIG suffered serious problems as it was not possible to add layers at all. ArcGIS Explorer errored when adding the DRG on multiple attempts, and also errored when adding the USGS shapefile of quadrangle boundaries. ILWIS was not considered as the package apparently duplicates all data into its own internal format rather than using data on the file system natively.

Although ArcExplorer passed the Base Case test its status of no longer being in development was deemed an issue for going forward, therefore, ArcExplorer was not included in the Extended
Case evaluation.

Extended Case

The remaining packages, MapWindow, OpenJUMP, QGIS and, uDig were evaluated with a more extensive and fully populated oil and gas workflow and related dataset. Several different machines were used, another XP box, a 32-bit Vista machine and a high-end QuadCore 64-bit Windows 7 box with 8Gb of RAM. This diversity in available computing resources is somewhat more reflective of real world conditions.

The base map and E&P related shapefiles, such as, contracts, basins, fields and wells were extracted from EDIN-GIS, and others were downloaded from a public domain datasource, Natural Earth. The well dataset consisted of more than 66,000 wells in the vicinity of northwestern South America. Fields, contracts, basins and wells from Central America and northern South America were selected out of larger datasets and committed to shapefile. One of the raster backdrops was a 660Mb TIFF from Shaded Relief. A similar raster from the same website was a large georeferenced JPEG. Also used as a test of raster backdrops was the WorldSat MrSid cloud-free earth image which is in ESRI Data & Maps DVDs.

The goal of the Extended Case evaluation was to simulate a relatively standard workflow common in the natural resources industry. Oil and gas workflows generally involve large datasets, backdrop rasters and/or rasters of interpreted work and ad hoc geometries lending useful additional information to a project. Typical usage of GIS datasets involves storage and use of spatial data stored in a GCS and projected-on-the-fly as needed. The question remaining is to what extent a free or no cost mapping package could emulate the behavior of vetted commercial packages used for mapping in the natural resources arena.
The area of interest was restricted to an area in and around Colombia. All spatial datasets were in a GCS. However, one of the datasets in ArcGIS was projected to Colombia Bogota Zone, a Transverse Mercator projection using the Bogota datum, and was included in the Extended Case evaluation.

Several factors were used in the Extended Case evaluation. Primarily, the goal was to examine the extent to which each application could reproduce or simulate the functionality of several features found in ArcGIS which are sometimes taken for granted:

· Load large vector and raster datasets, including large TIFF, JPEG and MrSid rasters.
· Definition Query, workflow for emulation of the functionality
· Symbology, Class Breaks/Unique etc.
· Labeling
· Identify
· Project-on-the-fly

The text describing the Extended Case is presented as a summary and a series of observations about the application, and its behavior in both the Base Case and Extended Case evaluations.

MapWindow

Extended Case Summary: MapWindow could load and display all the datasets in both Base and Extended Cases. Emulation of a definition query is by writing out a selected set from a query on a table. Using a plug-in it is possible to select shapes and write them out to a shapefile. Symbology is limited to simple primitives. Symbolic representation of class breaks is possible; however, colors cannot be picked from a ramp, but must be entered manually. Labeling is functional, and Identify works on selected layer in the Table of Contents. The app will project known layers to the layer of the dataframe with several options as to how to handle. MapWindow sets the map window as the projection of the first encountered dataset just as ArcGIS does. Projection organization and referencing is similar to that of ArcGIS.

Notes:

The program installs cleanly on all Windows variants, but does require the .NET 3.5 framework which is now common on most Windows machines, but might not be installed on older boxes.
Not possible to measure distances using decimal degrees using the provided plug-in, such measurements must be made in a projection.

There are useful and simple tools to construct shapes from a shapefile. Also has a built-in mechanism, via plug-in, to create a shapefile from a CSV, dBase or Excel input.

Has bookmarks and can import/export them.
Has a layout applet and simple tools to generate and manage marginalia.
The program does not have spatial selection capability, however, this functionality is exposed in the underlying API.

Setting one label on a shapefile of multipart polygons is time-consuming, but produces good results.

The product can be configured to self-populate the application and datasets.
Has GPX and GPS support, but not tested.

OpenJump

Extended Case Summary: OpenJump could not load the vector datasets without generating a memory heap error, however, it was able to recover without crashing. A definition query can be simulated in OpenJump, however, it is a multi-step process, as the attribute query tool is relatively simplistic. Symbolization is easy and met the criteria of the evaluation. OpenJump became unstable with just some of the vector layers loaded. Labeling and Identify on one high-resolution polygonal world shapefile could not be tested due to memory constraints.

Notes:

OpenJump does not perform project-on-the-fly for any datasets. Further, OpenJump appears geared for work strictly in a projection as the map scale using basic GCS datasets were consistently misreported. Also, when in-memory datasets are committed to the file system, a projection file is not generated by the application, therefore, any attempt to use that dataset in another mapping application requiring a PRJ file would require a manual step to create one.
An optional extension to perform coordinate transformations failed on an XP box and terminated the program when attempted on a Windows 7 box.

OpenJump will not display georeferenced JPEGs.
The application has an option for display of synchronized window which may be valuable in busy map displays.
The application is the only one of the four with any significant spatial query operators, although QGIS has some primitives. OpenJump can perform attribute queries as well as spatial queries. The output of these operations are in-memory datasets which can be persisted to shapefile using ‘Save Dataset As’.
The program does not honor selected sets in processing. Rather the datasets must be subsetted by another mechanism to a file or in-memory dataset for additional processing.
The only cartographic output supported by the application is to a JPEG, PNG or SVG file.
OpenJump does not have bookmark capability.

QGIS

Extended Case Summary: QGIS would install on Vista only under Administrator mode, and installed in Windows 7 only by running the installer in XP compatibility mode. QGIS could load and display all the datasets in both Base and Extended Cases. Emulation of a definition query is by writing out a selected set from a query on a table to a shapefile. Labeling is a weakness of the application. Multipart polygons have all entities labeled. Even geometries dissolved by its own geoprocessing tool will have all of its independent geometries labeled.

Notes:

QGIS can install and operate off of a USB stick. This was tested by installing it on a USB on a test box then executing QGIS from the USB on a clean, reimaged Windows 7 machine.
QGIS and OpenJump were the only applications to have significant spatial operations.
Only MapWindow and QGIS have a specific applet for handling cartographic output. The presentation of this functionality is similar to, but substantially limited from, that of an ArcGIS layout.
The app does allow for project-on-the-fly and has a bookmark capability internal to its project.
QGIS has extensive support for a number of third-party mapping applications, although the limits in this were not tested. It did, in fact, have no problem reading an ESRI Personal GeoDatabase v9.3.1.
QGIS can be used as a front-end to the more powerful GRASS GIS application. This functionality was not tested and the QGIS application which was tested was the ‘No-GRASSSetup’ version.
QGIS will on occasion generate non-fatal RTTI, run-time type, errors. The only solution for this was an application restart.
Has GPX and GPS support, but this functionality was not tested.

uDig

Extended Case Summary: uDig could not load all the raster datasets into the project without a memory error, however, it was able to load all the vector data. uDIG had memory problems with the introduction of the large TIFF file, and aborted abruptly in Windows 7 after this. Apparently, uDIG does not support georeferenced JPEGs. Emulation of a definition query is by writing out a selected set from a query on a table to a shapefile. The application can do quantile classification and will assign colors from a color ramp selector. The symbolization GUI experienced quite a few errors; however, this in itself did not cause the application to terminate ungracefully. Labeling is quite simplistic as the application skips labeling anything which may overpost. Identify is somewhat clunky as it is necessary to click twice to determine which Identify to invoke and twice to obtain the information about the entity clicked. The app will project known layers to the projection of the dataframe, however, the app was so taxed at this point it stopped responding and had to be killed.

Notes:

The application builds an EPSG database locally at startup and may require an internet connection to install a Java runtime environment at invocation. uDIG has good interaction between the table and map. However, query capability appears to be quite limited.
The application also lacks any spatial selection capability beyond a graphic rubberband box.
uDIG does not have the direct capability of saving a project file, although it does provide this functionality using a wizard. In addition, the program saves its state upon exit.
Application supports bookmarks internal to its project environment.

SUMMARY

MapWindow rated highly in terms of data handling, ease of use and functionality within the scope of the evaluation. QGIS despite its quirks with installation in non-XP environments has a number of utilities and functionality desirable in an application for users in natural resources.