Unlocking data inaccessibility through Web-mapping

1.1   Introduction

 

The study will take a look at a rapidly evolving technology that may serve as a tool for improving accessibility and dissemination of data. The technology of Web-mapping or Internet-mapping is investigated as per user-friendliness and rapid access to data.

 

The relevance of the Internet to current thinking is analysed, especially the prospect of reduced manual intervention for map-generation and the elimination of ownership of dedicated map applications. Further analysis would include a look into the upsurge of Web-mapping initiatives with particular reference to the commercial-academic push-pull factors. Different Web-mapping technologies, from low tech/low cost to high tech/highly expensive systems, are looked into. The study will also investigate data accessibility issues and will assess the socio and technological barriers that hinder the same accessibility.

 

For the purpose of this study, a test case, based on the Malta 1995 Census[1] is planned, which study aims to investigate whether Web-mapping can serve as a transporter for both tabular and mapped data. The case study attempts to develop a working model of a Census Web-mapping service, through the use of either GeoMedia Web Map[2] as a high tech-system or an alternative dual system using low-tech image-maps and interactive maps in conjunction with ArcExplorer[3]. Pre- and post-study questionnaires shall be carried out in order to assess whether potential users see the necessity for further development of the application.

1.2   The Internet and Web-mapping Technologies

 

Current GIS developments on the Internet as a medium to convey data are based on the desire to access an international network which users are familiar with, can be accessed from any location, when required without the constraints of owning a proprietary software for map-generation and data analysis. The web is relevant for GIS data owners since data can be stored where users can find it, in a format that they can read irrespective of platform or system they are using, as well as using a common and inexpensive if not free browser.

 

Developments of applications for the Web have been impressive. Software vendors have been quick to jump on the bandwagon and initially provided applications based on existing GIS with web code wrapped around them[i] (Limp, 1999a). These first applications were very limited in scope and allowed for only a small number of users. Later developments included the ability to handle threading where many users can access the data since each request is separated in packets and worked upon independently by server software. The current applications make use of libraries of alternatives that do the work of a full GIS package, based on Object-Oriented technology[ii] (Hardy et al, accessed 1999).

 

   1.2.1   Is the issue relevant?

 

The Internet emerged during the mid-1990s as a dominant force especially with the launching of the World Wide Web, even though its real origin can be traced to the 1960s[iii] and was restricted to military and academic institutions (Kristula, 1997).

 

No study of phenomenology[iv] (Natason et al, 1994) ever encountered such a rapidly evolving occurrence, with some arguing over its own existence[v]. Society itself has yet to come to grips with the ramifications of the web, how it will change the media, communications, research, study, warfare and a multitude of other societal relationships. Sociology itself will have to radically shake itself to face this new continuum. The Internet has already become an integral part of society, and some returns decidedly evident: on-the-second journalism, on-line bookstores and auctions, real-time worldwide videoconferences, global research, remote consultancies during medical intervention and home publishing [vi] (Plewe, 1997).

 

With the inception of the World Wide Web (WWW), Internet use exploded and today it is quite unclear who is pushing what. In turn this highlighted the push-pull debate on whether academic institutions are pushing the development of Web-mapping or vendors are pulling it along. White papers such as those produced by Intergraph[vii],[viii] (developers of GeoMedia WebMap and GeoMedia Web Enterprise), have contributed to this issue along with developments by organisations such as ESRI[ix] (MapObjects, Arcview IMS), AutoDesk[x],[xi] (Mac Gillavry, 1999),  (MapGuide), and MapInfo[xii] (Mapxsite, Mapxtreme). Some kind of stabilization may be on the horizon through initiatives such as the OpenGIS Web-mapping Testbed RFT[xiii],[xiv] (Ferreira et al, 1998) and the TC/211 Geographic Information/Geomatics Committee of the International Standards Organisation[xv],[xvi] (Rowley, 1999 – Levinsohn, 2000).

 

To date, some good on-line GIS applications have been built, from the early stages of Internet development, examples of which include the early XEROX PARC Map Viewer[xvii] (launched June 1993) and the Virtual Tourist[xviii] (launched January 1994). Since then, a multitude of map generating sites has been created[4]. These include some specialised sites where Web-mapping is highly advanced both in relation to generic issues as well as in such specific areas as is Census Web-mapping. An example of the former can be found at Delaware Data[xix], which has a section comprising a transport interactive map linking the user to live traffic cameras as well as having links to Census information. Sites specific to the topic under study include that of the Detroit Empowerment Zone Census Tracts Selection[xx] with its highly interactive map by means of links to census tracts and data.

 

Both public and private organisations have seen the potential of Web-mapping and are jumping to the challenge to produce better sites, however there is Web-mapping and Web-mapping. Whilst it is desirable to have a fully-fledged GIS on the web, some applications do not necessarily need the full-works; some may need simple pre-formatted bit-maps, and others may need network based collaborative GIS where remote users share data in real time.

 

 

   1.2.2   Web-mapping applications

 

A range of Web-mapping applications is available for the developer with pre-drawn bitmaps at one end. Bitmaps are very low-end solutions where little interaction can occur. At the other end of the range is the sophisticated solution mainly used for Intranet mapping but has the same potential for Internet mapping, where maps are generated on a full GIS application that resides on the web-server. Such a system is called Internet server-based GIS. Other Web-mapping solutions[xxi] (Greenwood, 1999) exist such as:

 

o    Image maps - the most common Web-mapping sites in active service. Though pre-drawn, they allow a certain level of interaction as users can jump to datasets, other maps, html links, images, etc. through hot-links embedded in the map itself.

o    Map-generating sites – generally simple search engines that generate maps on the fly. Road maps or location finders feature a lot in this area[xxii]. Users type in the name of a street and a map of the particular location is provided.

o    Third-party mapping - a company creates the maps for a client and injects the same maps on the client’s web-page, but each map is linked to the company’s site, where the map actually resides. The user is not aware of the migration.

o    Java applets – pre-written mini-applications that are becoming quite popular as a method to display data and maps[xxiii],[xxiv]. Applets allow interactivity and work best on vector data where individual features are made active.

o    Mini GIS plugins or Active X components - downloadable components that are installed on the client’s side and interact with the downloaded data from the web-server, reducing data traffic and downloading.

o    CGI Scripting (Common Gateway Interface) - platform-independent protocols that interpret user messages and translate them into instructions for server action returning selective maps to the user.

 

But what do these applications imply for such studies as the current one? The main preoccupation is that in the process of developing the research study, the technology employed would be obsolete, although the medium may be the same or upgraded, especially with the much-flaunted Internet2[xxv] development. This is where OpenGIS moves in. A GIS dataset with no allegiance to any proprietary software or system will definitely be the one to survive in such a rapidly evolving system. The user need not have a high-end software at his/her end, the map/data generation tool resides in the servers’ end and queries are resolved there. This implies that a user with a basic browser can have access to data generating power from the comfort of one’s home or office..

 

In order to analyse whether Web-mapping does help to offset problems of accessibility and facilitates ease of use of data, a case-study shall be carried out on the Census 1995 of the Maltese Islands.

1.3   Availability and Accessibility of Census data in Malta

           

   1.3.1   1842-1967 Censuses

 

The Censuses[5] in Malta were officially launched in 1842, following the introduction of the Census in Great Britain in 1801 and covered the demographic as well as the social and economic characteristics of the population. The latest take in 1995 was authorised by Legal Notices[xxvi],[xxvii],[xxviii] in terms of Section 3 of the Census Act of 1948[xxix] and employed the “De Jure” enumeration method[xxx].

 

   1.3.2   1985 Census

 

Most Census data in Malta is in analogue (paper) format. Documents from the census takes of 1861[xxxi], 1871[xxxii], 1891[xxxiii], 1901[xxxiv], 1911[xxxv], 1921[xxxvi], 1931[xxxvii], 1948[xxxviii], 1957[xxxix],[xl], and 1967[xli],[xlii],[xliii] are in this format[6]. Though no raw digital data exists for most of these Censuses, population data for 1957 and 1967 were inputted and presented in several formats, mainly as tables and population pyramids[xliv].

 

The next Census was taken in 1985[xlv],[xlvi]. The Central Office of Statistics (COS)[xlvii] and the University of Keele of the United Kingdom carried out the Census take. Volumes I and III were produced covering the demographic aspects and a demographic atlas respectively. Ironically, the Economic Activity section, Volume II was never produced, presumably due to financial constraints.

 

Data in digital format exists for the 1985 Census, however it is stored in an archaic format and the gatekeepers could only use specialised software (SPSS[7]) to access it. In addition to the limitations imposed by the software, a greater barrier to accessibility is encountered: the human and policy one. In order to access the data a complex system of requests to the gatekeepers has to be made[xlviii]. The data requester has to ask for permission to access the data from the COS who are the owners of the Census. Once approval is given, the researcher has to approach another organisation (M.I.T.T.S. Ltd.[8]) for their approval to access the software. M.I.T.T.S. claim to be the guardians of the data and as such no researchers other than themselves can analyse the data for confidentiality reasons. Thus, access to the data and outputs from the same data is severely restricted and can be a daunting task.

 

   1.3.3   1995 Census

 

The 1995 Census saw a changing aspect to the dissemination of data to the public. Organisations such as the PA that had collaborated with the COS and provided pre-Census services (such as maps for enumeration areas, questions for the questionnaire[xlix], etc) were requested to forward their requests for data from the Census office. The data requested could only be given up to specific levels of aggregation to safeguard the confidentiality clause of the Census Act. Table 1 outlines the aggregation levels.

 

Table 1: Aggregation Levels for Census outputs, 1995

 

Prior to producing the digital data, COS published a preliminary report for parliament in March 1996[l], followed by population[li], fertility[lii], nuptiality and households[liii], education and economic activity[liv], migration[lv] and dwellings[lvi] publications. All publications were in analogue format and information was at Locality level and higher.

 

   1.3.4   The problem at hand

 

PA data was required in digital tabular format[lvii] (Appendix A), which requirement could not be accommodated, as “this was not possible at the time”. Following continuous communications, data was presented in *.lst files which is an SPSS output format. This entailed a process of data conversion into database format by initially cleaning it from artifacts and unwanted typos, then porting the text to a spreadsheet and aligning all the datasets into one table. A list was prepared as per datasets that have been converted and were used for the Web-mapping project (Appendix B). This was accompanied by a time-consuming exercise entailing the documentation of the steps taken through detailed lineages and error checking processes[lviii] (Appendix C).

 

Had the data been provided in *.dbf format all these problems should have been avoided. As such, even though Census data is made available, the accessibility issue is still far from solved. While the above indicates that data accessibility is to a certain extent still very limited and restricted, once the PA has the converted data the logical step would be to identify what steps the organisation needs to take in order to disseminate its data as widely as possible and in a user-friendly manner.

 

Diverse dissemination methods have been used to date, ranging from presenting the data in PowerPoint[9] presentation format (as in the case of the population pyramids), through the use of a spreadsheet-based output using Excel[10], to the preparation of a full-blown GIS based on MapInfo[11]. To add to the already complex situation, data presented to the PA also needs to be analysed as against other data in diverse formats from other organisations such as public domain information on income[lix] and the Household Budgetary Survey[lx].

 

What most users really look out for is a simple interface requiring as minimum intervention as possible. Few users are prepared to go the way of a steep learning curve in addition to acquiring expensive and high end software when they would rarely be using it to its full potential.

 

Once the problem has been analysed and the main options for dissemination covered, the next stage is to ascertain whether Web-mapping is a reliable medium on which to develop the Census service.

1.4   Could Web-mapping solve the problems of the Census in Malta?

 

Census data is to a certain extent an end in itself and cannot be updated until another Census is taken. This situation brings up a query whether there is a need for a full service based application.

 

On-line sites utilizing different web-mapping technologies show that Census data can be mapped using the Internet as the medium. These include: KINDS[12] with their dataMapper and cdv[lxi]; SEDAC’s[13] DDViewer and DDCarto[lxii]; SANDAG’s[14] Interactive Map[lxiii]; DemographicsNow’s[lxiv] Interactive Demo Site; Casweb’s[lxv] Web-based Interface; DVRPC[lxvi] Data Services[lxvii]’ Site; Georgia County’s Census Data Map[lxviii]; and the United Nations Popmap[lxix] project. These sites give evidence to the fact that Census-related databases can be brought together with mapping technology on the Internet in order to produce a live on-line analysis tool.

 

Whilst the availability of static image maps is already satisfactory for most Census studies, users in the PA identified further needs. A preliminary study in the organisation acknowledged that users want data that they can manipulate and analyse. This is consonant with what Charlton et al state is “a prerequisite for sensible geographical analysis, free from the tyranny of fixed arbitrary census geography”[lxx] (Charlton, 1995). One has to note that the 1985 Census maps[lxxi] were presented for a limited and specific number of variables. Should the 1995 data be made available on the web in the image map format each image has to be linked to the particular data file, generating a nearly unlimited number of outputs. This is especially more so since the data can be presented at different aggregation levels. Presenting data at National level is one thing, presenting it at Council (67 in number) and Enumeration Areas (800+) is another. This apart from image maps for every age group, job type, dwelling type, etc.

 

Therefore, a solution would be needed where maps could be generated as per request either through a full server-based GIS web-map solution or a dual system where the concept of image-mapping is shown in conjunction with an application that would allow the user to access the outputs across the web and generate maps at the users’ end as developed by Tantalus Communications[lxxii] through the use of free software as is ArcExplorer (Harder, 1998).

1.5   Resources Available

 

The study is based on the development of a Census Web-mapping Service for the Maltese Islands. For this scope, contact with diverse organisations has been established where hopefully one high-tech solution would be made available for use for this research.

 

Two options are available for the research strategy: either a full server-based GIS web-map solution (preferred strategy) or a dual system integrating image-mapping together with a map-generating and data querying application (alternative strategy). Figure 1 depicts the two options.

 

Figure 1: Web-mapping Development Options

 

The first option involves GeoMedia WebMap, which is a full server-based GIS web-map solution that would allow users to access the Census data over Intranets and the Internet and generate maps as well as tabular data. Since this development requires a steep learning curve from the developer, and the development itself is still a new concept, the aim of this section is to develop a working prototype of the Census. However, should developments prove otherwise, a more extended service would be attempted with more datasets put on-line. For this option, programming support, mainly Java and VB scripting would have been made available from PA colleagues from IT and Research Units.

 

One point to note at this stage concerns the fact that at first the development will be presented on an Intranet and following the post-development questionnaire feedback, decisions would be taken on whether to develop the service further.

 

Should difficulties be encountered in the first option, depending on software provision or development of the service, an alternative option was prepared. This option involves the setup of a hybrid system that brings together a low-tech image-map system with a web-map application (ArcExplorer), which would be installed at the user’s end and accesses data and maps over the Internet. This dual system would allow the user to view examples of the maps that can be generated by Census category (ex. Gender and age, dwelling stock). Should they need further information, ArcExplorer would facilitate access to data in order to generate maps or data queries as they deem fit from the large number of combinations available. ArcExplorer would also allow the user to export data and maps for external use.

 

Each of these options is a viable investigative tool to deduce whether Web-mapping would facilitate data accessibility. Though it would be desirable to develop a full-server based application, the dual system would still prove a challenge.

1.6   Conclusion

 

The study shall endeavor to explore how Web-mapping caters for the needs of users and analyses whether it is the best solution to present Census data on the web for dissemination. The main aim of this research is to make the data available using the most appropriate solution, through an easily accessible medium. The research will aim to produce a working Census Web-mapping prototype that the users will then analyse and from their feedback decisions can be taken whether the technology can be further developed as per other datasets within the Planning Authority.

 

In conclusion, the study shall endeavor to investigate whether Web-mapping is a viable solution to all the accessibility issues that Census researchers encounter to date in Malta.

1.7   Research Structure

 

The Thesis is structured as follows, whereas Figure 2 gives a graphic description of the perceived stages:

 

Figure 2: Research Development Stages

 

Chapter 1 – Introduction: unlocking data inaccessibility through Web-mapping

 

·         The Extended Outline forms the basis of the first Chapter, mainly the issue of data accessibility and the integration of Census and Web-mapping.

 

 

 

 

Chapter 2 – Web-mapping and accessibility – how are Web-mapping technologies being utilised?

·         are the main current uses the only technology for the future? An analysis of the past, present and future: a case-by-case approach. What are the problems of accessibility vis-à-vis data.

·         What Web-mapping technologies exist: full-blown software, imagemaps, cgi, hotlinks, database availability, live data, etc. How do they facilitate access to date?

·         Census can be produced as an ongoing process for a place as small as Malta through the integration of data on-line from different.

·         What other options are there as against Web-mapping?

 

 

Chapter 3 – Case-study: Web-mapping the 1995 Census of Malta

 

·         What does the study involve?

·         How was the data made available, how was it converted, what was the process to produce the current state of data?

·         What software was used and why was that option taken?

·         Is it worth the effort and will users benefit from the data made available: are requests continuous in the organisation under study.

·         Describe how the data works (as a manual and also compare costs of setting up as against use).

·         Make data available for use by the University and other researchers.

·         Create Help file for users – either as a separate document or as an HTML file.

·         Does the product relate to the general ideas set out above?

·         Critique of method and how the results can be improved.

 

 

Chapter 4: - Getting Feedback: - Pre-Exercise & Post-Exercise Questionnaire

 

·         Who are the test recipients?

·         Pre-development questionnaire. Who was consulted prior to initiating the study? What were their initial requests for data availability? Why do they need the data? What are their options for accessibility as regards the current data? What was their initial feedback?

·         Post-development questionnaire. What do they think of the service? Do they approve of it? Does it facilitate their access to data?

·         What are their comments on future accessibility and whether they would go for a more complete web-service with on-line data and real-time querying capabilities. Considering the fact that the Census is a static dataset what can be added to it in order to provide extensions to the product for updated use?

·         Weighting the outcome: Pros and cons resulting from their comments, which comments should be retained for further development or discarded and why?

 

 

Chapter 5 – Conclusions

 

·         What are the pros and cons of such an exercise?

·         What are the points that result in improvement of the service? Has the exercise in Web-mapping proved its worth? Is data really accessible to all? Is software too expensive? Is the technology developing too fast to make sense for the web-map developer? How does the user fit in all this?

·         Does Census availability as a web-map alleviate problems of access to data? Has accessibility been vindicated?

·         Would going for a more extensive service be ideal, especially as regards to doing away with multiple-licenses and user-registrations from proprietary software.

·         Recommendations

·         Recommendations resulting from the study

·         Recommendations for further study/research

 

Footnotes

[1]  Test case will be based on Census data presented to the Malta Planning Authority (PA) by the Central Office of Statistics (COS)