An intro to linked and open local gov data

Linked and open data in local government
an overview and informal discussion piece on the role of the LGA Group
August 2010

How to read this paper: Donʼt be put off by the length of this paper. You can read the
introduction and 11 key points and stop. Or you can read on for more detailed explanation
and expansion of the work that the LGA group is intending to support. Council chief
executives will be written to shortly explaining more about the offer and inviting
involvement from key staff. We invite your participation and discussion in the Local Open
Data Community of Practice.

Introduction
Key points about open and linked data

1. A basic explanation of open and linked data
2. The benefits of open and linked data
3. Whatʼs happening now
4. Barriers to linked and open data
5. What councils need to do now and how the LGA Group is helping

Introduction

The previous Labour government made significant strides in opening up government data.
The ground breaking data.gov.uk initiative prompted the release of huge numbers of
central government held data sets, some of which are centrally-collated local government
data (e.g. national indicator sets and some financial data). In one of the last policy
announcements of the previous government, Gordon Brown pledged to release even more
public data along with other initiatives in Building Britainʼs Digital Future. Although not
mentioned in the headlines, local government data was a key element in the expansion of
open data and the personalised portals to services for citizens.

In the run-up to the election the Conservatives were supportive of the open data initiative.
Under the coalition government, some of the initiatives such as promised funding for the
Web Sciences Institute have been slashed or abandoned, but their commitment to open
data remains strong. Not only are they pushing for more data to be published, but they
are also pressing for new types of data to be released, for example salaries, job titles and
expenditure in both central and local government. A letter from the Prime Minister on 31
May 1, set out the governments initial ambitions for open data which includes publishing
local government expenditures greater than £500.

Open data and local government:

1http://www.number10.gov.uk/news/statements-and-articles/2010/05/letter-to-government-departments-on-
opening-up-data-51204

The data.gov.uk project which had a public launch in January has focused on the ʻeasiestʼ
sets of data to expose. But it was always the plan to include local government data. The
Local Government Data Panel was established to support local government in releasing
data sets in a linked data format. On 4 June, greater clarity around the first tranche of open
local government expected was published on the CLG website 2. Shortly afterwards,
Secretary of State for CLG Eric Pickles indicated that legislation will follow if councils are
reluctant to share this information accessibly online. The LGA Group has now established
a Transparency Programme to support councils to achieve the minimum requirement and
to take advantage of the opportunities of linked and open data.

Some councils have been making strides with linked and open data, for example through
the Open Elections Data Project which encouraged councils to use ʻfreeʼ fixes to publish
local elections data. But for now, there are only a handful of examples. Most notably the
GLA which has been working across London, Warwickshire, Windsor and Maidenhead and
Lichfield District Council. Local government on the whole has not made open or linked
data a priority and is unlikely to be prepared to take advantage of the opportunities of
linked and open data.

Eleven key points about open and linked data

1. The UK has made amazing strides in opening up non-personal data held by government
in the last 12 months. Data.gov.uk has pushed up the global stakes in accountability
and transparency. But there is still al lot more to do in expanding and improving the
accessibility to government data and in making use of the data.
2. Most of attention so far has been on central government data, but local government
data is where the real gains can be made. The coalition government is putting pressure
on local government to open more data, not least the requirement that all expenditure
over £500 is made public within 6 months. Councils can do this the lazy way by
dumping lists into .pdfs (which is now seen as unacceptable), or we can grab the
opportunity to publish this openly as linked data which gives us amazing benchmarking
and efficiency opportunities.
3. Open data and linked data are not the same thing. Information can be made open, but
published in formats (like .pdfs or locked excel spreadsheets) that makes it pretty much
useless and difficult to combine with other data sets. Thatʼs not real transparency.
4. Linked data is the ability to add one data set to another data set and use it for further
applications - whatʼs called mashups. To do this we need to make some changes to the
way we present information and we need some naming standards - or ontologies.
These arenʼt necessarily expensive changes, but they do require some skill and effort.
5. Linked data does not have to be open data. Public services would benefit tremendously
from using linked data formats. It means that we could stop spending resources on data
aggregation and start spending it on analysis and action. Linked data can be used in
secure settings to help partners share personal, sensitive or commercial information on
performance and resources and help better target those in need or areas for
improvement.
6. There are lots of potential benefits to linked and open data. Most of them fall under
enhancing democracy and accountability, making useful applications for citizens and
consumers and better use of information by government itself. All of them have huge
efficiency opportunities. Weʼre only at the beginning of realising the benefits.

2 http://www.communities.gov.uk/news/corporate/1606882

7. Few councils in the UK are really doing very much with open and linked data. But we
can look to these early leaders and examples in the US for potential benefits and key
lessons.
8. There are some substantial cultural and institutional barriers to opening up government
data. And we could easily fall into the trap of endlessly arguing about standards and not
just getting on with it. But if we work together, we can crack this and reap the rewards.
9. Several LG Improvement and Development programmes play into this space. In
particular, the Knowledge Hub and ESD-toolkit and Efficiency Exchange. We need to
do more to leverage these and ensure that weʼre supporting and capturing the benefits
across other programmes, too - including our work on enhancing democracy, supporting
efficiency, performance management and information exchange across partnerships.
We have an opportunity to play into this through the cross-LGA group Transparency
programme. For now, and necessarily, thatʼs concentrating on the £500 expenditure
issue, but we must take the opportunity to think more broadly.
10. Thereʼs an urgent need to support the sector in developing the technical skills. We can
support partners like Socitm and LeGSB to lead in this area. The LGA Group needs to
take the lead in demonstrating the benefits and arguing the case in plain language for
Members, senior managers across public services and among policy, performance and
efficiency leads.
11. This programme of support canʼt be carried out in the usual way. The open data
movement is a space occupied by passionate individuals - creative, brilliant, often
maverick and hungry for change. This is about changing the nature of the relationship
between government and citizens as it is about putting some numbers out there. Itʼs as
much about supporting and working with the market to provide useful applications as it
is about traditional market transactions and doing it ourselves. Those working within
institutions need to be able to work nimbly, flexibly and openly. We must communicate
early and often and be willing to not only listen to but converse and engage with a range
of voices, who can bring experience of leading change and the technical and creative
capability to turn raw data into useful stuff.

The rest of this paper expands on these points and provides a bit more contextual and
technical information.

1. Open and linked data - a very basic explanation

1.1 What is open data?

Open data is as much a philosophical movement as it is a programme of work. Itʼs the
idea that non-personal government data belongs to the people and should therefore be
freely available on the web and accessible for information and for re-use by citizens and
for commercial purposes. Not all open data is linked data. But if open data is published in
machine readable formats (e.g. as a regular web page rather than a .pdf), itʼs often
possible for it to be manipulated and used as linked data, through a process known as
ʻscrapingʼ.

1.2 What is linked data?

Linked data is data that is structured in such a way that data sets can be combined to
create something greater than the sum of its parts. Sometimes it also described as ʻthe
semantic webʼ. Sir Tim Berners-Lee, the inventor of the World Wide Web and chief
advocate of the semantic web, describes it as being a “web of data” the same way that the
Internet is currently a “web of documents” which uses standard language (HTML) to
provide sensible links between related topics which have a unique address - the URL.

This web of data will rely on a common framework (an ontology) to make links between
types of data (councils, locations, types of services, types of defects, etc) and unique
identifiers for unique ʻthingsʼ (URIs). It also usually requires an adjustment of the standard
language of web publication - changing HTML for XHMTL. But these are not things that
the average person needs to worry about and you wouldnʼt necessarily see a difference if
you saw the final web page. There are also certain technical fixes which make fairly
standard web pages into linked data - embedding standards and the structure of linked
data through something called RDFa. In many cases, itʼs perfectly acceptable to publish
open data as a CSV file, something which can be done from Excel, for example, or many
other data base programmes. And in other cases, itʼs necessary to publish web pages
which are really just for the machines to make sense of the data automatically. Those in
leadership functions in local government donʼt need to understand how they work exactly,
but only that they do need to work in order to achieve the benefits of linked data.

The terms linked data and open data are often used interchangeably, but they are not the
same thing. However, for open data to be at its most powerful it should be linked data.
Linked data doesnʼt have to be open; it can also be used in secure settings. For example
using this approach could help councils and their partners share information more
efficiently without exposing it to the public gaze.

2. The benefits of linked and open data

There are several key benefits:

2.1 Knowledge is power: Transparency and accountability - the concept that exposing
data in and of itself is a societal “good” and will support openness through the
democritisation of information. There is also the notion that transparency will change the
behaviour of government officials. For example, Windsor and Maidenhead have attributed
exposing carbon consumption data to the reduction of the councilʼs carbon footprint.

There are also cost savings which could be achieved by making public data open instead
of complying with costly and time-consuming FOI requests. Washington DC has
estimated that it has made considerable savings by taking this approach.

2.2 Making useful stuff: By exposing publicly held data sets and allowing commercial re-
use via a new kind of copyright license, developers can combine a variety of data sets - for
example public data sets, privately held data or crowdsourced information to create new
applications useful to citizens and consumers which can be accessed via the Internet or
the mobile web. It is believed that open data can have significant economic benefit,
supporting the development of creative and technical industries as well as benefits to
consumers.

We are beginning to see evidence of this in the UK, chiefly through the London Data Store
and there are numerous examples of this in the US, where several major cities (San
Francisco, Chicago, Washington, DC and others) are providing open data which has
spawned useful transport, fault reporting and other information services for citizens at little
to no cost to the city government itself. Warwickshire is experimenting with the same
approaches used by US cities by supporting contests to encourage developers to make
useful applications from their open data.

And by opening data, there are opportunities for councils to gather additional data from
citizens. For example, a mapped report of environmental defects could be augmented by
adding citizen reports of dead street lights, potholes, fly tipping etc which the council may
have missed, similar to the approach of the LoveLewisham site. Or they could avoid
duplicate citizen reports, if people know the council already has it in hand. In the US
some cities are already combining citizen data with public data. Chicago has done work
combining official crime statistics with citizen intelligence 3 (e.g. reports of ʻsuspiciousʼ
behaviour) and information from other departments (e.g. location of empty homes). San
Francisco has published information about all the trees on public property like parks and
has invited citizens to add information about other trees in the ʻurban forestʼ.

2.3 Better use of government data by government: although itʼs not widely mentioned,
public exposure of data sets means that government officials can more easily find and link
related data sets. This data does not need to be open, but if published in linked data
formats can be used to combine secure data with open data in private settings. This has
strong implications for LSP and Total Place approaches and could form a core element of
a de-centralised performance framework.

There are also significant cost savings which could be achieved. For example, councils
now subscribe to data aggregation services because itʼs a hugely labour intensive to pull
together various types of ʻopenʼ but not machine readable data. Some councils and
regions establish local information systems or regional data observatories which spend
considerable resources bringing data together from a variety of public sources. Some of
this expenditure could be avoided almost altogether or re-directed to analysis if both
agencies, government departments, councils and other local public service bodies
published non-personal data as linked and open data or protected data sets as linked
data.

3. Whatʼs happening now?

In response to Eric Pickleʼs statement on open data, Baroness Margaret Eaton
underscored the sectorʼs commitment to open data and the support of the LGA.4

"Local government is absolutely committed to the highest standards of transparency.
Councils have been leading the way in giving taxpayers real, detailed and vital information
about how their money is spent. All public bodies must be scrutinised for the spending
decisions they make, and the LGA will work with councils to pioneer an approach of
openness and accountability."

3 Cited in NESTAʼs Radical Efficiency publication: http://www.nesta.org.uk/home/assets/features/
radical_efficiency

4 http://www.communities.gov.uk/news/corporate/1606882

There are a number of initiatives around linked and open data in local government with
connections to the LG Improvement and Development and the LGA Group.

3.1 The Local Government Data Panel5 was established in December 2009 to
encourage greater open data sharing and to create a platform for open data exchange. In
practice, this probably means data.gov.uk. It includes Tim Allen from LGAAR as well as
local government representatives. Francis Maude has since established a Government
Transparency Board with an overall remit to consider government data and transparency
issues, but it does not have any local government representation.

3.2 The Local e-Government Standards Body has begun work to establish standards
which can help local government agree a common ʻdataʼ language. A first workshop to
identify issues around standards was held in late May. A number of naming conventions
already exist, but in some cases there are duplicates (e.g. various identifiers for councils -
some from ONS and other less sophisticated naming approaches) or standards are
inconsistently applied (e.g. Integrated Public Service Vocabulary). This workshop explored
the possibility of establishing a working group, but terms of reference and membership
have not yet been identified, but the work is likely to include the development of a primer
and identification of skills and standards gaps. Follow-up workshops have taken the
practical approach further - including developing an outline for creating linked data around
local government expenditure data.

3.3 There is a workstream within the Knowledge Hub to identify and integrate a number
of sources of data within the platform to integrate performance, contextual and efficiency
data into the conversational space. The Knowledge Hub wonʼt be entirely dependent on
linked data, but open linked data will be used where available, pulling in information from
sources like data.gov.uk and london.data.gov.uk - among other data sources. It will openly
encourage linked data - as itʼs a more effective way of integrating data. The Knowledge
Hub will also seek to have linked data standards “baked in” to the core of the way it
operates - using accepted and emerging standards as the spine for the way that data is
organised and presented and so that data can be re-used by others.

3.4 The ESD-Toolkit is using linked data principles and accepted data conventions for its
cost-to-serve model - which is a sector derived way of describing the functions and
purpose of local government. And they are exploring ways to publish case studies using
linked data standards. This is a tremendous opportunity to link stories with data. ESD-
Toolkit has also developed a proposal to help councils meet the requirements of publishing
open data with support around data cleansing and enabling the data to be published as
linked data.

3.5 The Open Elections Data Project led by Chris Taggart of Openly Local and a
member of the LGDP encouraged councils to experiment with opening up local election
data and sharing it in linked data format. This project was supported by the LGA and LG
Improvement and Development and practitioners shared information and advice through
the CoP platform. Key findings from that project have been published in the Local Open
Data community of practice and on the data.gov.uk blog.

3.6 The Knowledge Hub programme and the CLG unit which is supporting the opening up
of local government data held a workshop of key stakeholders toward the end of June to
explore how we can work together to support the development of open and linked data.

5 http://blogs.cabinetoffice.gov.uk/digitalengagement/post/2009/12/07/Local-Government-Data.aspx

This event was set up before the expenditure data announcements, but the day focused
on these events.

4.Barriers to linked and open data

4.1 Fear of transparency - including the fear that errors in the data sets will be exposed.
4.2 Technical considerations and skills: There is a learning curve and not everyone has
the skills to make linked data available on council websites.
4.3 Data hugging - some people derive considerable power from holding information which
they just donʼt want to let go of.
4.4 Outsourced websites. Some councils have outsourced their websites which means
that making even small technical adjustments to the way that information is published
could have considerable cost.
4.5 Lack of a widely accepted business case and evidence of benefits in a language that
most policy leads or chief executives can understand.

5. What councils need to do now and how the LGA Group is helping

The bar has already been set with specific requirements for councils to publish
expenditure data and contracts information and there is an expectation that more council
data should be released. Councils are being encouraged to publish information in
machine readable formats and will be encouraged to publish this as linked data. If this
doesnʼt happen voluntarily; it may be imposed. Some members of the LGDP are already
thinking of recommending legislation to force councils to open up more data in accessible
formats. Some data activists are relishing the possibility of bombarding councils with FOI
requests if they do not voluntarily publish expenditure data by the January deadline.

As councils and other local public service bodies are both required and seek to put more
information in the public domain there are several areas where the LGA Group can help.
The LGA Group has now established a working group under the strategic leadership of
Sara Williams and Tim Allen to meet the transparency challenge. This programme is being
managed by Ian Carbutt and contains representation from ESD (Tim Adams), Knowledge
Hub (Ingrid Koehler), as well as LGAAR staff (Gesche Schmid, Jonathan Evans) and
partners from SOCITM, CIPFA and LeGSB.

5.1 Explaining the benefits. We need to explain the benefits, risks and opportunities in
clear and non-technical language to senior managers and members in local government.
As local government faces tough financial times, we need to explain how data can be
shared more effectively quickly and cheaply and what the benefits will be to councils, their
partners and citizens.
5.2 Supporting quick wins: some of the work around open and linked data will take time
and does require technical skill and staff resources. However, some things - like
publishing public documents as web pages rather than (or alongside) PDFs or publishing
spreadsheets in more readable formats rather than locked excel spreadsheets takes
almost no time and costs nothing. There are also a few basic data standards which could
turn the publishing of council expenditure from a burden into a benchmarking opportunity.

5.3 Celebrating the successes and transferring the learning: Through our existing
knowledge sharing resources, we should celebrate councils that are opening up data for
efficiency, improvement and enhanced citizen experience.
5.4 Anticipating the problems: Opening up local government data is bound to bring a raft
of difficulties. Many fear being overwhelmed by queries in response to the publication of
expenditure data in a time when councils are already stretched. There will be significant
challenges for Members, comms teams and others as local government enters a new age
of transparency. But others have gone before and we can learn from the challenges that
cities in the US and early adopters in the UK have already overcome and share those
lessons more widely with the sector.
5.5 Technical support - by working with SOCITM and others and through existing
programmes (Knowledge Hub, ESD-Toolkit, and others) and communities of practice we
can support practitioner communities to agree the basic standards and reference data sets
for linked data and to share the skills required for technical staff.
5.6 Working with Government, LeGSB, CLG and the Local Government Data Panel to
make the transition to open data easier and to make the benefits of open and linked data
more obtainable by putting pressure on the Ordnance Survey to relax licensing conditions
on derived geographic data which is still a major hindrance to making useful data open to
the public without complex and expensive licensing.
5.7 Working with the National Archives on having a simple opendatacommons.org
license (similar to the established CreativeCommons Licenses) that can be applied to
otherwise unlicensed local authority data and where possible issuing our own content
under similar licenses.
5.8 Providing solutions for councils that are unable to publish open data - lessons
from the Open Elections project indicate that there are technical, structural as well as skills
issues that will have to be overcome. ESD-toolkit is offering a platform for publication,
some data cleansing and linked-data support.

Ingrid Koehler, 3 August 2010

Appendix 1: Further “reading” and resources

Background

• Tim Berners-Lee on the next Web | Video on TED.com explains what linked data is and
the potential rewards of linking data.
• Tim Berners-Lee: The year open data went worldwide | Video on TED.com a reflection
on open data initiatives including examples from the US and UK.
• The Silent State: Secrets, Surveillance and the Myth of British Democracy by Heather
Brooke. This book by the reporter who helped break the parliamentary expenses
scandal through a series of FOI requests is a clear exposition of the philosophy of open
data activists and includes chapters on the early work of open data developers using
local data to make useful information applications for citizens.

Resources
• Data.gov.uk blog: http://data.gov.uk/blog
• Local data wiki - established by two council officers and a social entrepreneur to explain
and support the development of open local data : http://localdata.pbworks.com/

• Local Open Data community of practice, facilitated by SOCITM Insight and started by
Chris Taggart a member of the Local Government Data Panel
• Openly Local - a useful resource and index around local government open data,
developed by Chris Taggart.
• I have been collecting interesting examples of open and linked data in local government
through the http://www.socialgov.posterous.com blog under the data tag.

An intro to linked and open local gov data

More Related Content

What's hot

Viewers also liked

Similar to An intro to linked and open local gov data

More from Ingrid Koehler

An intro to linked and open local gov data