The MESH project aimed to promote harmonised production and use of marine habitat maps covering the North West European seas by adapting and enhancing previous achievements in this area. It involved the collation of habitat mapping information across several countries, including UK, Ireland, Netherlands and Belgium. The Confidence Assessment Tools were created by the MESH Project help to assess the suitability of maps to their intended purpose.
2007
Questions this practice may help answer
- How suitable is a map for its intended purpose from an analytical perspective?
- How good is the remote sensing, ground truthing and data interpretation for this map?
- Can you quantify the confidence level and/or accuracy of a particular seabed habitat map?
Implementation Context
If a user needs to make an important decision based on the information contained in a habitat map, how can they gain a level of confidence that the map is reliable? If you have commissioned a survey, how can you be certain that the resulting habitat map will match your expectations? When a cartographer prepares maps, how can they convey its accuracy and limitations to the users?
Habitat mapping is defined by the MESH Project to be:
“Plotting the distribution and extent of habitats to create a map with complete coverage of the seabed showing distinct boundaries separating adjacent habitats.”
Seabed habitat maps show the distribution of habitats by interpreting physical data layers, often derived from remote sensing, using biological information about habitats obtained from direct sampling and observation of the seabed. Only a small proportion of the seabed can be observed or sampled and the complete coverage of habitats is inferred from the association between the physical habitat data and the seabed samples so the final maps predict the distribution of seabed habitats.
Visualising the spatial distribution, quality and quantity of seabed resources is fundamental to our understanding of marine ecosystems and our ability to manage human activities to deliver effective sustainable development and maintain marine ecosystem function. Maps have a wide range of applications in management, planning, policy and research and form an important and integral part of management information systems.
Aspects / Objectives
- To assess the usefulness of habitat maps depending on their intended purpose and application
- Habitat maps provide a fundamental information layer for spatial and strategic planning and help support sustainable use of seabed resources. Increased confidence in this data assists decision makers
Method
An important part of the MESH process has been to focus on the integrity of existing maps and to encourage an honest / open approach to assessing the suitability of each map to the intended purpose. The project has developed a ‘confidence assessment methodology’ where the evaluation process addresses three main questions:
1. How good is the remote sensing?
2. How good is the ground truthing?
3. How good is the data interpretation?
These questions were selected because MESH promotes the creation of habitat maps through the interpretation of remote sensing data and ground truthing data. The maps on the MESH webGIS are presented with confidence ratings so that end-users can determine their adequacy for decision-making. In addition, future survey effort can be strategically directed to areas where maps have relatively low confidence or no map coverage.
To help users assess the confidence of a map, the MESH Project developed a systematic approach using a multi-criteria questionnaire. The MESH Project collated many maps produced for different purposes, ranging over many years, employing many different techniques and from a variety of sources. In many areas the maps overlap and users need to know the relative confidence they should have in these maps. The problem of map confidence is multifaceted and any assessment runs the risk of being very subjective and dependent on the person undertaking the exercise. Clearly, if some comparison is to be made between maps, then the assessment should be as objective as possible.
The MESH Partners decided that a confidence assessment system should be devised and the confidence factors stored as new metadata elements so that they are accessible together with the discovery metadata describing the map. The metadata already compiled by the project lacked sufficient detail to make objective decisions about the confidence of various factors. The purpose of the MESH Confidence Assessment was to visualise an overall confidence score on the MESH webGIS using study outlines, and to link these outlines to the full set of scores so that the assessment process remains transparent. The overall scores allow rough comparisons to be made between maps whereas the full set of scores enables users to identify why one map may have scored more highly than another. A scoring system based on a multi-criteria approach also allows survey planners to anticipate the effect of changing various survey parameters on the overall performance of a survey. In other words, it may help as a planning tool. Similarly, some criteria will also be useful to cartographers to determine which factors will increase or decrease the confidence a user may place in a map.
The MESH Confidence Assessment scheme is a compromise between being comprehensive and being easy to understand and use. Many criteria have undoubtedly been left out and the exact scores suggested for each map may be challenged. The system is not designed to identify subtle differences between maps, but rather to give a simple and robust assessment. The exact score for any one field could be debated, but the overall score is little affected by tweaking the individual scores for the fields. Although the way in which a multi-criteria scoring approach is designed and operates is open to criticism, at least it is also open to inspection because the decision points are established and guidance is given to standardise scoring carried out by different individuals.
Main Outputs / Results
The MESH Confidence Assessment scheme has been built into two applications, each of which is best suited to a particular type of confidence assessment.
For assessing multiple maps, we suggest using the MESH Confidence Scoresheet (a MS Excel Spreadsheet) for ease of data entry and comparison between maps.
For a more interactive tool which is best suited to the assessment of a single map, use the MESH Confidence Tool which was built as a Flash application. This interactive tool allows the user to see the effect of changing individual scores and weightings on the overall score. For those interested in the methodology but who will not be carrying out confidence assessments, the factors and scoring system are set out in the MESH Confidence Assessment Guidelines. The scoring guidelines are built into each application for quick reference.
Transferability
Whilst these confidence assessment tools have been created for assessing seabed habitat maps, they could be used to assess other habitat maps.
Responsible Entities
Joint Nature Conservation Committee (JNCC)
IFREMER – Institut français de recherche pout l’exploitationde la mer
University of Gent
Marine Institute
IMARES (formerly Alterra-Texel)
TNO - Netherlands Organisation for Applied Scientific Research
Centre for Environment, Fisheries and Aquaculture Science (CEFAS)
Agri-Food and Biosciences Institute (AFBI) (formerly DARD)
Natural England
Envision Mapping Ltd
National Museum Wales (NMW)
British Geological Survey (BGS)
Costs / Funding Source
INTERREG IIIB North-West Europe area
Total project cost: € 8.106.708
Contact
project
meshatlantic.eu (project[at]meshatlantic[dot]eu)