KEMBAR78
EDF2012 Nigel Shadbolt - Transparency and Open Data | PPT
Transparency and Open Data:
Why Bother?
Being also a discourse on some lessons learnt


Professor Nigel Shadbolt FREng

7th June 2012

European Data Forum, Copenhagen

                                                Nigel Shadbolt
open the data…
and the applications follow…
at all scales…
around the world…
leading to…

• Improved transparency &
  accountability
• Better Public Services

• Participation… people enhance,
  improve, capture the data

• Improved efficiency

• Creation of social value
• Economic value, innovation and
  growth
Lesson: Open Data…
should be non-partisan
“Our ambition is to                      “We are determined to go further in breaking
become the world leader                  down the walled garden of government,
in open data, and                        using technology and information to provide
accelerate the                           greater transparency…”
accountability revolution
that the internet age has
made possible..
                                                     “My aim is clear: to make
                                                     the provision of data (with
 “Greater transparency across                        reasonable formats and
 Government is at the heart of our                   licenses) a routine task of
 shared commitment to enable the                     public administrations.”
 public to hold politicians and public
 bodies to account
Lesson: Open Data…
needs a steady stream of
        success
Lesson: Open Data…
is not a technical problem
It is about changing behaviour

UK Public Data
Principles
However…



★   Put your data on the Web with an Open Licence
★★        Make it available as structured data
★★★ Use open, standard formats
★★★★      Use URIs to identify things
★★★★★     Link your data to other people’s data
Create, Use and Share URIs


• National digital
  infrastructure being built

• URIs for schools, roads,
  bus stops, post codes,
  admin boundaries...

• Some of the data links
  across and connects
  other data together

• Key data link points exist
Create, Use and Share URIs


• National digital
  infrastructure being built

• URIs for schools, roads,
  bus stops, post codes,
  admin boundaries...

• Some of the data links
  across and connects
  other data together

• Key data link points exist
Create, Use and Share URIs

• Schools

• Post-codes

• Government posts

• Ministers

• Bus stops

• :

• :
Lesson: Open Data…
if it doesn’t scale it will fail
Datasets on data.gov.uk over time
Open Data – the network effect

• Network effects
   – Accessibilitly
   – Popularisation
   – Centreless
   – Timeliness
   – Convergence




                                 Courtesy Mathew Hurst
Lesson: Open Data…
needs location, location, location
www.ordnancesurvey.co.uk/opendata




See also: Linked data http://www.ordnancesurvey.co.uk/opendata/linkeddata.html
the connective tissue of open data
24
Lesson: Open Data…
release data that matters
UK Reported Crime Data
• A tough data set to
  release

• All reported crime each
  month

• By crime type

• At the street level

• Police now use the
  apps themselves
From Schools to Arterial Surgery
Lesson: Open Data…
must have a balanced ecosystem
Source Deloitte LLP
Research &
                                                                h
                                                         Academic c
• The Network Effect                                      Cluster
                                                                r

   •   for data, people and organisations
• A Focal Point
   •   coordinate, convene, link and
       integrate
                                            Business
                                             Cluster
                                                   r
                                                    s
                                                         ODI
                                                           I            Public Sector
                                                                             c
                                                                          Clusterr
                                                                                    r

• The Power of Open
   •
                                                        Tech City
                                                           h y
       Standards, Data, Licences, SW
• Open Data as a Platform for
                                                        Interna onall
                                                          Agenciess
   •   Government                                          Cluster
                                                                 r
   •   Business
   •   Research
“The vision is to establish the Open Data Institute as a world leading
      centre to innovate, exploit and research the opportunities for the UK
                 created by the Government’s Open Data policy.”

•   Business Innovation

•   Public Sector Innovation

•   Training the Open Data Generation

•   Developing and Researching Open Data

•   Developing Open Data Standards and Policies

•   Advisor to Government & International Collaboration
OGD 2.0
• We haven’t worked out
  how to close the
  feedback loops

• Publish out

• Acquire or Enhance

• Write Back
Lesson: Open Data…
  needs Business
Economic Value, Innovation and Growth
39
40
Lesson: Open Data…
   isn’t everything
Varieties of Data

                    • Big - Small
                    • Public – Private
                    • Open - Closed
                    • Personal - Non-Personal
                    • Anonymous - Identified
                    • Aggregate - Individual




                                      © 2012 Deloitte Global Services Limited
midata

         Nigel Shadbolt
Personal Data Asset Revolution…
Lesson: Open Data…
 will need defending
Challenges

• Infrastructure

• Quality

• Interpretation

• Security and Privacy

• Evidence of impact

• Sir Humphrey 2.0
A sample of “data hugging” excuses
•   It’s held separately by n different organisations, and we can’t join it up

•   It will make people angry and scared without helping them

•   It is technically impossible

•   We do not own the data

•   The data is just too large to be published and used

•   Our website cannot hold files this large

•   We know the data is wrong

•   We know the data is wrong, and people will tell us where it is wrong

•   We know the data is wrong, and we will waste valuable resources inputting the corrections people
    send us

•   People will draw superficial conclusions from the data without understanding the wider picture

•   People will construct league tables from it

•   It will generate more Freedom of Information requests

•   It might be combined with other data to identify individuals/sensitive information

•   It will cost too much to put it into a standard format

•   Our IT suppliers will charge us a fortune to do an ad hoc extract
A sample of “data hugging” excuses
•   It’s held separately by n different organisations, and we can’t join it up

•   It will make people angry and scared without helping them

•   It is technically impossible

•   We do not own the data

•    • All these sometimes have some truth in them
    The data is just too large to be published and used

•    • website they rationalise
    OurOftencannot hold files this large the official fear of the unknown

•    • know the data is wrong
    We Data owners need to be helped through

•    • know the data & precedent are your it is wrong
    We Example is wrong, and people will tell us wherefriends

•    • know first is wrong, and we will waste valuable resources not licence
    We For the dataphase, compromise on data inputting the corrections people
    send us

•   People will draw superficial conclusions from the data without understanding the wider picture

•   People will construct league tables from it

•   It will generate more Freedom of Information requests

•   It might be combined with other data to identify individuals/sensitive information

•   It will cost too much to put it into a standard format

•   Our IT suppliers will charge us a fortune to do an ad hoc extract
Lesson: Open Data…
keep reminding them
Open Data and the Power of Open
never one driver nor one reason


•Open Licences

•Open Standards

•Open Data

•Open Data Principles

•Open Participation

•Open Minds

EDF2012 Nigel Shadbolt - Transparency and Open Data

  • 1.
    Transparency and OpenData: Why Bother? Being also a discourse on some lessons learnt Professor Nigel Shadbolt FREng 7th June 2012 European Data Forum, Copenhagen Nigel Shadbolt
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
    leading to… • Improvedtransparency & accountability • Better Public Services • Participation… people enhance, improve, capture the data • Improved efficiency • Creation of social value • Economic value, innovation and growth
  • 7.
  • 8.
    “Our ambition isto “We are determined to go further in breaking become the world leader down the walled garden of government, in open data, and using technology and information to provide accelerate the greater transparency…” accountability revolution that the internet age has made possible.. “My aim is clear: to make the provision of data (with “Greater transparency across reasonable formats and Government is at the heart of our licenses) a routine task of shared commitment to enable the public administrations.” public to hold politicians and public bodies to account
  • 9.
    Lesson: Open Data… needsa steady stream of success
  • 12.
    Lesson: Open Data… isnot a technical problem
  • 13.
    It is aboutchanging behaviour UK Public Data Principles
  • 14.
    However… ★ Put your data on the Web with an Open Licence ★★ Make it available as structured data ★★★ Use open, standard formats ★★★★ Use URIs to identify things ★★★★★ Link your data to other people’s data
  • 15.
    Create, Use andShare URIs • National digital infrastructure being built • URIs for schools, roads, bus stops, post codes, admin boundaries... • Some of the data links across and connects other data together • Key data link points exist
  • 16.
    Create, Use andShare URIs • National digital infrastructure being built • URIs for schools, roads, bus stops, post codes, admin boundaries... • Some of the data links across and connects other data together • Key data link points exist
  • 17.
    Create, Use andShare URIs • Schools • Post-codes • Government posts • Ministers • Bus stops • : • :
  • 18.
    Lesson: Open Data… ifit doesn’t scale it will fail
  • 19.
  • 20.
    Open Data –the network effect • Network effects – Accessibilitly – Popularisation – Centreless – Timeliness – Convergence Courtesy Mathew Hurst
  • 21.
    Lesson: Open Data… needslocation, location, location
  • 22.
    www.ordnancesurvey.co.uk/opendata See also: Linkeddata http://www.ordnancesurvey.co.uk/opendata/linkeddata.html
  • 23.
  • 24.
  • 26.
  • 27.
    UK Reported CrimeData • A tough data set to release • All reported crime each month • By crime type • At the street level • Police now use the apps themselves
  • 31.
    From Schools toArterial Surgery
  • 32.
    Lesson: Open Data… musthave a balanced ecosystem
  • 33.
  • 34.
    Research & h Academic c • The Network Effect Cluster r • for data, people and organisations • A Focal Point • coordinate, convene, link and integrate Business Cluster r s ODI I Public Sector c Clusterr r • The Power of Open • Tech City h y Standards, Data, Licences, SW • Open Data as a Platform for Interna onall Agenciess • Government Cluster r • Business • Research
  • 35.
    “The vision isto establish the Open Data Institute as a world leading centre to innovate, exploit and research the opportunities for the UK created by the Government’s Open Data policy.” • Business Innovation • Public Sector Innovation • Training the Open Data Generation • Developing and Researching Open Data • Developing Open Data Standards and Policies • Advisor to Government & International Collaboration
  • 36.
    OGD 2.0 • Wehaven’t worked out how to close the feedback loops • Publish out • Acquire or Enhance • Write Back
  • 37.
    Lesson: Open Data… needs Business
  • 38.
  • 39.
  • 40.
  • 41.
    Lesson: Open Data… isn’t everything
  • 42.
    Varieties of Data • Big - Small • Public – Private • Open - Closed • Personal - Non-Personal • Anonymous - Identified • Aggregate - Individual © 2012 Deloitte Global Services Limited
  • 43.
    midata Nigel Shadbolt
  • 44.
    Personal Data AssetRevolution…
  • 45.
    Lesson: Open Data… will need defending
  • 46.
    Challenges • Infrastructure • Quality •Interpretation • Security and Privacy • Evidence of impact • Sir Humphrey 2.0
  • 47.
    A sample of“data hugging” excuses • It’s held separately by n different organisations, and we can’t join it up • It will make people angry and scared without helping them • It is technically impossible • We do not own the data • The data is just too large to be published and used • Our website cannot hold files this large • We know the data is wrong • We know the data is wrong, and people will tell us where it is wrong • We know the data is wrong, and we will waste valuable resources inputting the corrections people send us • People will draw superficial conclusions from the data without understanding the wider picture • People will construct league tables from it • It will generate more Freedom of Information requests • It might be combined with other data to identify individuals/sensitive information • It will cost too much to put it into a standard format • Our IT suppliers will charge us a fortune to do an ad hoc extract
  • 48.
    A sample of“data hugging” excuses • It’s held separately by n different organisations, and we can’t join it up • It will make people angry and scared without helping them • It is technically impossible • We do not own the data • • All these sometimes have some truth in them The data is just too large to be published and used • • website they rationalise OurOftencannot hold files this large the official fear of the unknown • • know the data is wrong We Data owners need to be helped through • • know the data & precedent are your it is wrong We Example is wrong, and people will tell us wherefriends • • know first is wrong, and we will waste valuable resources not licence We For the dataphase, compromise on data inputting the corrections people send us • People will draw superficial conclusions from the data without understanding the wider picture • People will construct league tables from it • It will generate more Freedom of Information requests • It might be combined with other data to identify individuals/sensitive information • It will cost too much to put it into a standard format • Our IT suppliers will charge us a fortune to do an ad hoc extract
  • 49.
  • 52.
    Open Data andthe Power of Open never one driver nor one reason •Open Licences •Open Standards •Open Data •Open Data Principles •Open Participation •Open Minds

Editor's Notes

  • #2 From Data to Decisions: The Power of Information in the Age of the Web of Data This talk will review the technology and trends that underpin our ability to derive intelligence from the data deluge we all face.  It will consider how a systems approach to the Web, its technical and social components, can help.  The role of open data, crowd sourcing and distributed problem solving will be discussed as they relate to your organization.
  • #12 Cholera Bacteria MSRA Bacteria June 9, 2012
  • #23 OS Open Data Overview of our OS Open Data product portfolio. More details obtained through following the links: www.ordnancesurvey.co.uk/opendata www.ordnancesurvey.co.uk/opendata/linkeddata.html
  • #36 UNCLASSIFIED
  • #51 Tom, Gavin and Dan made a paper. A paper where all of the content relates to your postcode. It tells you all about the local area centered around you and would make a lovely welcome greeting to any new arrival. It is a cornucopia of data from allotment waiting time to the time taken to travel to central London to information on local NHS and refuse/recycling centers.