KEMBAR78
Osm techniques and developemnt | PDF
OSM
                            Techniques and Development of
                                OpenStreetMap (OSM)

                                              at




                                        Dongpo Deng
                                   dongpo@iis.sinica.edu.tw

                                         Feb. 7. 2012


Tuesday, February 7, 2012
Forbidden city 2012-01




                                 Snapshot on 2012-1-20
Tuesday, February 7, 2012
Tuesday, February 7, 2012
OSM and me
           •       Do mapping in central TW and eastern
                   NL
           •       Edit ‘Map Feature’ (zh-tw)
           •       Host Taiwanese mapping parties
           •       My future work for OSM
                 •          Lobby for open geo-data
                 •          Establish TW local chapter of OSM
                 •          Do researches on OSM
                 •          Participate to SotM 2012
Tuesday, February 7, 2012
The major components
                         of OSM
                    • Portal sites   • Backend databases
                    • Editors        • Rendering systems
                    • Data sources   • Wiki documents
                    • OSM APIs       • Mapper communities

Tuesday, February 7, 2012
Go to map the world




Tuesday, February 7, 2012
Go to map the world

                            • OpenStreetMap (OSM) is non-for-
                              profit foundation
                            • OpenStreetMap (OSM) is a
                              collaborative project to create a free
                              editable map of the world
                            • Shared by OdbL 1.0 (Changed from
                              By-SA 2.0 since 2012)

Tuesday, February 7, 2012
Grassroots mapping
                    • Wiki-style editing environment
                    • Open License, Open Geo Data
                    • Volunteered Geographic Information




Tuesday, February 7, 2012
OSM can be used for...

   •       Map browsing,
   •       Map rendering,
   •       Searching,
   •       Reverse geocoding,
   •       Exporting,
   •       Online edition.



Tuesday, February 7, 2012
A simple way to
                             contribute data

                • GPS traces
                • Geotagged Photos
                • Paper and pen
                        (walking-papers.org)




Tuesday, February 7, 2012
As a OSM mapper, you not only
                upload data but also DRAW the map

               • Draw the map via uploaded GPS traces,
                       arial and satellite images, etc
               • Edit the data attributes
                • Name, e.g. street, building
                • Street type
                • POI
                • Traffic artifact ... ,etc
Tuesday, February 7, 2012
Mappers drawing Taipei




Tuesday, February 7, 2012
Architecture of OSM




Tuesday, February 7, 2012
Map Editors




Tuesday, February 7, 2012
Front-end editors
                    •       Online
                            •   Potlatch (Popular!)
                            •   Mapzen
                            •   Amenity Editor
                    •       Offine
                            •   JSOM (Popular!)
                            •   Merkaartor
                            •   Plugins QGIS

                       See more info http://wiki.openstreetmap.org/wiki/Editors
Tuesday, February 7, 2012
Polatch2




Tuesday, February 7, 2012
Mapzen




       mapzen.cloudmade.com
Tuesday, February 7, 2012
Amenity Editor




    http://ae.osmsurround.org/ae/index
Tuesday, February 7, 2012
JSOM




Tuesday, February 7, 2012
Merkaator




            http://merkaartor.be/
Tuesday, February 7, 2012
OSM Plugins for QGIS




Tuesday, February 7, 2012
Mobile Editors
                    • Apple iOS
                     • iPhone
                    • Symbian
                     • Nokia
                    • Andriod
            See more details http://wiki.openstreetmap.org/wiki/Software

Tuesday, February 7, 2012
Data resources:
                             GPS traces




Tuesday, February 7, 2012
Data resources:
                             GPS traces




Tuesday, February 7, 2012
Data source: Online
                                 Mapping systems
                    •       Oline mapping systems and open geo-data
                            resources
                            •   Yahoo! areal : way, landuse...
                            •   Landsat : main way, large landuse...
                            •   WMS services: OpenArialMap


                    •       Don't copy other map if the geodata contains
                            copyright issues!

Tuesday, February 7, 2012
Open GeoData to OSM
                    •       CC-licenced geo-data resources directly import to
                            OSM
                            •   AND (Automotive Navigation Data) : Netherlands,
                                India and China
                            •   TIGER data on USA
                            •   Corine Land Cover (Estonia, Romania, Hungary and
                                France)
                            •   Ordnance Survey (UK)
                            •   County of Brest, France
                            •   National parks (FR)
                            •   ...
Tuesday, February 7, 2012
Backend




Tuesday, February 7, 2012
Backend database: PostgreSQL
                •       OSM doesn’t use geometry type of PostgreSQL
                        and geospatial functions of PostGIS.
                •       OSM API v0.6 is used for the main central OSM
                        DB running on servers.
                      •     http://wiki.openstreetmap.org/wiki/API_v0.6
                •       Osmosis is used to populate a more general
                        PostgreSQL/PostGIS DB from Planet.osm dump
                        file.
                      •     http://wiki.openstreetmap.org/wiki/Osmosis

Tuesday, February 7, 2012
Database schema


                                  Text
                                   Text

                                          Relation & old data

              User
                                          Data

Tuesday, February 7, 2012
Tuesday, February 7, 2012
Tuesday, February 7, 2012
Tuesday, February 7, 2012
Geometry model
    Node
    Way
      Open ployline
            Closed ployline
             Area
     Relation
     Tag

    See detail info http://wiki.openstreetmap.org/wiki/Data_Primitives
Tuesday, February 7, 2012
Map feature
      •       Key / value
            •       higway = motorway
            •       name =
            •       ref = Fwy3
            •       maxspeed=110
      •       Recommended usage
            •       (wiki:Map Features)
      •       Consensus (Tagwatch, OSMdoc)
      •       Homogeneity : data reuse
      •       Requirement adjust
Tuesday, February 7, 2012
Rendering system




Tuesday, February 7, 2012
Mapnik
                                    www.mapnik.org


            •       OSM uses Mapnik to render 256 X
                    256 pixels tiles
            •       osm2pgsql is used to convert OSM
                    data to postgreSQL for Mapnik
                    rendering
            •       Map sytles are controlled by
                    stylesheet (XML) files.
            •       Useful tutorial! https://github.com/
                    mapnik/mapnik/wiki/MapnikTutorials

Tuesday, February 7, 2012
Mod tile
                    • Mod tile is a system to serve tiles, for
                            example to use within a slippy map.
                    • The implement of Mod file is split into two
                            ports
                            • mod_tile is a custom Apache module
                              which is responsible for serving tiles and
                              requests the rendering of tiles
                            • renderd - A rendering daemon which calls
                              mapnik and writes the map tiles to the
                              filesystem
Tuesday, February 7, 2012
tile@home
            • Tiles@home (short: T@H or tah) is a distributed
                    program to render Osmarender maps.
            • T@H has a server software, Tahngo (generation
                    2), running at the Tiles@home website, which get
                    requests to render tiles from updated mapdata.




Tuesday, February 7, 2012
Browsers




Tuesday, February 7, 2012
Slippy map
                    •       The slippy is an Ajax component. JavaScript runs in the
                            browser, which dynamically requests tiles from the
                            server in the background (without reloading web page)
                            to give a smooth map browsing experiences.
                    •       The implementation of slippy map is mostly provided
                            by OpenLayers
                    •       The tiles rendering usually comes from two resources
                            •   Mapnik tile rendering
                            •   Osmrender (tiles@home) tile rendering

                                       See details http://wiki.openstreetmap.org/wiki/Develop
Tuesday, February 7, 2012
Wiki documents




Tuesday, February 7, 2012
OSM community
                    •       Three main factors: wiki, mailing lists, mapping party




                                                                source: http://vaes9.codedgraphic.com/




Tuesday, February 7, 2012
Applications/Services
                          based on OSM
                             techniques


                            See details http://wiki.openstreetmap.org/wiki/
                                    List_of_OSM_based_Services
Tuesday, February 7, 2012
Crisis Map
   • shahanafoundation.org       • Ushahidi.org




Tuesday, February 7, 2012
opencyclemap.org




Tuesday, February 7, 2012
histOSM.org




Tuesday, February 7, 2012
Real-time train map
                             http://kubus.mailspool.nl/spoorkaart/




Tuesday, February 7, 2012
Humanitarian Response
                      and OSM


Tuesday, February 7, 2012
OSM Humanitarian Data Model




Tuesday, February 7, 2012
OSM in Haiti




                                                   Souce: Maron 2010

                 Before earthquake   7 days after earthquake


Tuesday, February 7, 2012
Researches on OSM



Tuesday, February 7, 2012
OSM participants




Tuesday, February 7, 2012
Number of Publications
                            indexed by Google Scholar




Tuesday, February 7, 2012
Some research topics on OSM
                    •       data quality, ontologies, top-down process (Muki
                            Haklay)
                    •       3D models and routing applications (Zipf et al.)
                    •       Neocartography and geovisualization (Chilton,
                            O’Brien)
                    •       Semantic, ontologies (Bishr, Antoniou)
                    •       Technical application (Mooney)
                    •       Trust and general quality (van Exel)



Tuesday, February 7, 2012
Types of research with OpenStreetMap

                        •   Muki Haklay (2011) summarized three types
                            •   With OSM - Working close to OSM community
                                to identify research needs and follow them
                                (Wiki Research Ideas page)

                            •   About OSM Learning about OSM, and the
                                community as part of VGI. Comparing it to other
                                projects

                            •   Using OSM–Using the data set to create new
                                applications and explore scientific issues
      Muki Haklay (2011) State of the Map EU - OpenStreetMap and GIScience research, State of the Map
      (SotM) 2011

Tuesday, February 7, 2012
Important open questions
      •       Cultural differences and the impact on mapping

      •       Impact of imports, and the rate and nature of updates of well
              mapped areas

      •       Spatial and temporal patterns of engagement and
              contribution, role of ‘slient’ mappers

      •       ‘Tyranny of place hypothesis’ and the impact of specific
              individual on mapping an area

      •       Internal culture and impact on gender, exclusion and long
              term engagement

      •       Legal studies - license use cases and application

      •       Usability (see Weber and Jones 2011)
                                                           Source: Muki Haklay (2011)
Tuesday, February 7, 2012
Limitations of academic research / collaboration

            •       Short attention span - the incentives are for novelty,
                    not routine
            •       Publications are important, so research ideas are
                    welcomed
            •       Academic institutes can provide power and network
                    (up to a point), not money
            •       Enthusiastic students are available at specific time cycle
            •       Across the world, match funding for business, so
                    introductions needed
            •       Academia careers have thematic focus
                                                            Source: Muki Haklay (2011)
Tuesday, February 7, 2012
Code of engagement for OSM research

               •      Rule 1– Join a mapping party.

               •      Rule 2 – Read OSM wiki, books, blogs, and mailing lists

               •      Rule 3 – Explore the data.

               •      Rule 4 – Open assess.

               •      Rule 5 – Open knowledge.

               •      Rule 6 – Be a critical friend.

               •      Rule 7 – Teach.

                                                               Source: Muki Haklay (2011)
Tuesday, February 7, 2012
Code of engagement for OSM research

                    •       Maintain links with the OSM community –it will
                            pay off and will help you to identify new
                            research directions.
                    •       Also maintain links with in the VGI research
                            community –even if the term is awkward, the
                            research is valuable.
                    •       Explore comparisons and parallels –it’s
                            important to learn what is going on in other
                            projects.
                                                               Source: Muki Haklay (2011)

                      http://wiki.openstreetmap.org/wiki/Research
Tuesday, February 7, 2012
Remarks
                    •       OSM is a wiki-style map based on open sources
                            and open data.
                    •       OSM is a collaboration community. All
                            participants are volunteers.
                    •       More and more services and applications use
                            OSM data and techniques.
                    •       OSM is a good instance of VGI. Many researchers
                            around the world are working on OSM.


Tuesday, February 7, 2012
Question!




                                         State of the Map 2012 in Tokyo

                                    Taiwan Mapping party on Feb. 18, 2012



                            Creative Commons License
                            This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License

Tuesday, February 7, 2012

Osm techniques and developemnt

  • 1.
    OSM Techniques and Development of OpenStreetMap (OSM) at Dongpo Deng dongpo@iis.sinica.edu.tw Feb. 7. 2012 Tuesday, February 7, 2012
  • 2.
    Forbidden city 2012-01 Snapshot on 2012-1-20 Tuesday, February 7, 2012
  • 3.
  • 4.
    OSM and me • Do mapping in central TW and eastern NL • Edit ‘Map Feature’ (zh-tw) • Host Taiwanese mapping parties • My future work for OSM • Lobby for open geo-data • Establish TW local chapter of OSM • Do researches on OSM • Participate to SotM 2012 Tuesday, February 7, 2012
  • 5.
    The major components of OSM • Portal sites • Backend databases • Editors • Rendering systems • Data sources • Wiki documents • OSM APIs • Mapper communities Tuesday, February 7, 2012
  • 6.
    Go to mapthe world Tuesday, February 7, 2012
  • 7.
    Go to mapthe world • OpenStreetMap (OSM) is non-for- profit foundation • OpenStreetMap (OSM) is a collaborative project to create a free editable map of the world • Shared by OdbL 1.0 (Changed from By-SA 2.0 since 2012) Tuesday, February 7, 2012
  • 8.
    Grassroots mapping • Wiki-style editing environment • Open License, Open Geo Data • Volunteered Geographic Information Tuesday, February 7, 2012
  • 9.
    OSM can beused for... • Map browsing, • Map rendering, • Searching, • Reverse geocoding, • Exporting, • Online edition. Tuesday, February 7, 2012
  • 10.
    A simple wayto contribute data • GPS traces • Geotagged Photos • Paper and pen (walking-papers.org) Tuesday, February 7, 2012
  • 11.
    As a OSMmapper, you not only upload data but also DRAW the map • Draw the map via uploaded GPS traces, arial and satellite images, etc • Edit the data attributes • Name, e.g. street, building • Street type • POI • Traffic artifact ... ,etc Tuesday, February 7, 2012
  • 12.
  • 13.
  • 14.
  • 15.
    Front-end editors • Online • Potlatch (Popular!) • Mapzen • Amenity Editor • Offine • JSOM (Popular!) • Merkaartor • Plugins QGIS See more info http://wiki.openstreetmap.org/wiki/Editors Tuesday, February 7, 2012
  • 16.
  • 17.
    Mapzen mapzen.cloudmade.com Tuesday, February 7, 2012
  • 18.
    Amenity Editor http://ae.osmsurround.org/ae/index Tuesday, February 7, 2012
  • 19.
  • 20.
    Merkaator http://merkaartor.be/ Tuesday, February 7, 2012
  • 21.
    OSM Plugins forQGIS Tuesday, February 7, 2012
  • 22.
    Mobile Editors • Apple iOS • iPhone • Symbian • Nokia • Andriod See more details http://wiki.openstreetmap.org/wiki/Software Tuesday, February 7, 2012
  • 23.
    Data resources: GPS traces Tuesday, February 7, 2012
  • 24.
    Data resources: GPS traces Tuesday, February 7, 2012
  • 25.
    Data source: Online Mapping systems • Oline mapping systems and open geo-data resources • Yahoo! areal : way, landuse... • Landsat : main way, large landuse... • WMS services: OpenArialMap • Don't copy other map if the geodata contains copyright issues! Tuesday, February 7, 2012
  • 26.
    Open GeoData toOSM • CC-licenced geo-data resources directly import to OSM • AND (Automotive Navigation Data) : Netherlands, India and China • TIGER data on USA • Corine Land Cover (Estonia, Romania, Hungary and France) • Ordnance Survey (UK) • County of Brest, France • National parks (FR) • ... Tuesday, February 7, 2012
  • 27.
  • 28.
    Backend database: PostgreSQL • OSM doesn’t use geometry type of PostgreSQL and geospatial functions of PostGIS. • OSM API v0.6 is used for the main central OSM DB running on servers. • http://wiki.openstreetmap.org/wiki/API_v0.6 • Osmosis is used to populate a more general PostgreSQL/PostGIS DB from Planet.osm dump file. • http://wiki.openstreetmap.org/wiki/Osmosis Tuesday, February 7, 2012
  • 29.
    Database schema Text Text Relation & old data User Data Tuesday, February 7, 2012
  • 30.
  • 31.
  • 32.
  • 33.
    Geometry model Node Way Open ployline Closed ployline Area Relation Tag See detail info http://wiki.openstreetmap.org/wiki/Data_Primitives Tuesday, February 7, 2012
  • 34.
    Map feature • Key / value • higway = motorway • name = • ref = Fwy3 • maxspeed=110 • Recommended usage • (wiki:Map Features) • Consensus (Tagwatch, OSMdoc) • Homogeneity : data reuse • Requirement adjust Tuesday, February 7, 2012
  • 35.
  • 36.
    Mapnik www.mapnik.org • OSM uses Mapnik to render 256 X 256 pixels tiles • osm2pgsql is used to convert OSM data to postgreSQL for Mapnik rendering • Map sytles are controlled by stylesheet (XML) files. • Useful tutorial! https://github.com/ mapnik/mapnik/wiki/MapnikTutorials Tuesday, February 7, 2012
  • 37.
    Mod tile • Mod tile is a system to serve tiles, for example to use within a slippy map. • The implement of Mod file is split into two ports • mod_tile is a custom Apache module which is responsible for serving tiles and requests the rendering of tiles • renderd - A rendering daemon which calls mapnik and writes the map tiles to the filesystem Tuesday, February 7, 2012
  • 38.
    tile@home • Tiles@home (short: T@H or tah) is a distributed program to render Osmarender maps. • T@H has a server software, Tahngo (generation 2), running at the Tiles@home website, which get requests to render tiles from updated mapdata. Tuesday, February 7, 2012
  • 39.
  • 40.
    Slippy map • The slippy is an Ajax component. JavaScript runs in the browser, which dynamically requests tiles from the server in the background (without reloading web page) to give a smooth map browsing experiences. • The implementation of slippy map is mostly provided by OpenLayers • The tiles rendering usually comes from two resources • Mapnik tile rendering • Osmrender (tiles@home) tile rendering See details http://wiki.openstreetmap.org/wiki/Develop Tuesday, February 7, 2012
  • 41.
  • 42.
    OSM community • Three main factors: wiki, mailing lists, mapping party source: http://vaes9.codedgraphic.com/ Tuesday, February 7, 2012
  • 43.
    Applications/Services based on OSM techniques See details http://wiki.openstreetmap.org/wiki/ List_of_OSM_based_Services Tuesday, February 7, 2012
  • 44.
    Crisis Map • shahanafoundation.org • Ushahidi.org Tuesday, February 7, 2012
  • 45.
  • 46.
  • 47.
    Real-time train map http://kubus.mailspool.nl/spoorkaart/ Tuesday, February 7, 2012
  • 48.
    Humanitarian Response and OSM Tuesday, February 7, 2012
  • 49.
    OSM Humanitarian DataModel Tuesday, February 7, 2012
  • 50.
    OSM in Haiti Souce: Maron 2010 Before earthquake 7 days after earthquake Tuesday, February 7, 2012
  • 51.
  • 52.
  • 53.
    Number of Publications indexed by Google Scholar Tuesday, February 7, 2012
  • 54.
    Some research topicson OSM • data quality, ontologies, top-down process (Muki Haklay) • 3D models and routing applications (Zipf et al.) • Neocartography and geovisualization (Chilton, O’Brien) • Semantic, ontologies (Bishr, Antoniou) • Technical application (Mooney) • Trust and general quality (van Exel) Tuesday, February 7, 2012
  • 55.
    Types of researchwith OpenStreetMap • Muki Haklay (2011) summarized three types • With OSM - Working close to OSM community to identify research needs and follow them (Wiki Research Ideas page) • About OSM Learning about OSM, and the community as part of VGI. Comparing it to other projects • Using OSM–Using the data set to create new applications and explore scientific issues Muki Haklay (2011) State of the Map EU - OpenStreetMap and GIScience research, State of the Map (SotM) 2011 Tuesday, February 7, 2012
  • 56.
    Important open questions • Cultural differences and the impact on mapping • Impact of imports, and the rate and nature of updates of well mapped areas • Spatial and temporal patterns of engagement and contribution, role of ‘slient’ mappers • ‘Tyranny of place hypothesis’ and the impact of specific individual on mapping an area • Internal culture and impact on gender, exclusion and long term engagement • Legal studies - license use cases and application • Usability (see Weber and Jones 2011) Source: Muki Haklay (2011) Tuesday, February 7, 2012
  • 57.
    Limitations of academicresearch / collaboration • Short attention span - the incentives are for novelty, not routine • Publications are important, so research ideas are welcomed • Academic institutes can provide power and network (up to a point), not money • Enthusiastic students are available at specific time cycle • Across the world, match funding for business, so introductions needed • Academia careers have thematic focus Source: Muki Haklay (2011) Tuesday, February 7, 2012
  • 58.
    Code of engagementfor OSM research • Rule 1– Join a mapping party. • Rule 2 – Read OSM wiki, books, blogs, and mailing lists • Rule 3 – Explore the data. • Rule 4 – Open assess. • Rule 5 – Open knowledge. • Rule 6 – Be a critical friend. • Rule 7 – Teach. Source: Muki Haklay (2011) Tuesday, February 7, 2012
  • 59.
    Code of engagementfor OSM research • Maintain links with the OSM community –it will pay off and will help you to identify new research directions. • Also maintain links with in the VGI research community –even if the term is awkward, the research is valuable. • Explore comparisons and parallels –it’s important to learn what is going on in other projects. Source: Muki Haklay (2011) http://wiki.openstreetmap.org/wiki/Research Tuesday, February 7, 2012
  • 60.
    Remarks • OSM is a wiki-style map based on open sources and open data. • OSM is a collaboration community. All participants are volunteers. • More and more services and applications use OSM data and techniques. • OSM is a good instance of VGI. Many researchers around the world are working on OSM. Tuesday, February 7, 2012
  • 61.
    Question! State of the Map 2012 in Tokyo Taiwan Mapping party on Feb. 18, 2012 Creative Commons License This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License Tuesday, February 7, 2012