Location-based hybrid urban environment for

intangible community heritage protection

Yang Bai
December 2022

Abstract
Making invisible elements that contribute to the construction of a community’s
memory and identity accessible, even to the community members themselves,
presents a challenge to heritage conservation efforts[1]. Digital and mixed set-
tings have the potential to assist in combining the tangible and intangible fea-
tures of historic structures and objects, according to recent work in the field
of digital heritage[2]. This article examines how intangible characteristics of a
location, such as historically and culturally significant events, may be made vis-
ible and put in dialogue with a physical space, as well as with human activities,
in a augmented reality environment. The research site is located in the Lower
Hill District around the Freedom Corner in Pittsburgh, Pennsylvania, which is
a monument and symbol of community discussion, unity, and civic engagement
in the city[3]. I describe the creation of a location-based augmented reality ap-
plication that overlays historical and cultural background information at Pitts-
burgh’s Freedom Corner and its surrounding historical area, taking a broad view
on community heritage as a spectrum of perspectives and activities generated
and operated by communities themselves[4][5]. It provides a real-world illus-
tration of how interactive augmented reality might assist the public in building
understanding through experiences that interactively overlay heritage informa-
tion within a physical landscape. It describes the interplay between historical
research and advancement towards the environment’s technical implementation.

Keywords
Intangible Heritage, Community Heritage, Location-based Augmented Reality,
Oral History, Freedom Corner, the Lower Hill District

1
Contents

1 Introduction 6
1.1 Research Question . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Background Research 10
2.1 Intangible Cultural Heritage . . . . . . . . . . . . . . . . . . . . . 10
2.1.1 Definition and Value Evaluation of Intangible Heritage . . 10
2.1.2 Intangible Heritage protection . . . . . . . . . . . . . . . . 12
2.2 Extended Reality for Intangible Heritage Protection . . . . . . . 13

3 Hypothesis 18

4 Method Statement 18
4.1 Case Study: Lower Hill District . . . . . . . . . . . . . . . . . . . 18
4.1.1 History and Community Assets of the Hill District . . . . 18
4.1.2 Urban Renewal Project in the Lower Hill . . . . . . . . . 20
4.1.3 Civic Right Activities around Freedom Corner . . . . . . 22
4.1.4 Heritage Site Selection and Evaluation . . . . . . . . . . . 22
4.2 Content Collecting . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2.1 Historical Images . . . . . . . . . . . . . . . . . . . . . . . 27
4.2.2 Online Oral History Collection . . . . . . . . . . . . . . . 28
4.2.3 Other audio resources . . . . . . . . . . . . . . . . . . . . 35
4.3 System Building . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.3.1 Pretest demo . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.3.2 Modeling and digital site building . . . . . . . . . . . . . 39
4.3.3 Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.3.4 Tracking and Registration . . . . . . . . . . . . . . . . . . 44
4.3.5 Location-based Asset Display . . . . . . . . . . . . . . . . 44
4.3.6 Audio Elements Settings . . . . . . . . . . . . . . . . . . . 46
4.3.7 Interaction Settings . . . . . . . . . . . . . . . . . . . . . 50
4.4 Experience Design . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.4.1 Content Presentation . . . . . . . . . . . . . . . . . . . . . 51
4.4.2 Design Scenarios . . . . . . . . . . . . . . . . . . . . . . . 52
4.4.3 User Flow Design . . . . . . . . . . . . . . . . . . . . . . . 54

5 Evaluation 56
5.1 User Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.1.1 Overall Process . . . . . . . . . . . . . . . . . . . . . . . . 56
5.1.2 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5.1.3 Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5.2 Data Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.3 Data Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2
 5.3.1 Multimedia Assets Display . . . . . . . . . . . . . . . . . 60
5.3.2 Understanding about the district and local heritage . . . 61
5.4 Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

6 Discussion 64
6.1 Intangible Heritage Representation via Multimedia . . . . . . . . 64
6.2 Design Implications . . . . . . . . . . . . . . . . . . . . . . . . . . 64

7 Future 66
7.1 Current Limitations . . . . . . . . . . . . . . . . . . . . . . . . . 66
7.1.1 Technological Limitations . . . . . . . . . . . . . . . . . . 66
7.1.2 Design Limitations . . . . . . . . . . . . . . . . . . . . . . 66
7.2 Future works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
7.2.1 Technological Improvements . . . . . . . . . . . . . . . . . 66
7.2.2 Design Improvement . . . . . . . . . . . . . . . . . . . . . 66

8 Conclusion 67

A Assets List 74

B User Test Responses 74

B.1 Interview Transcripts . . . . . . . . . . . . . . . . . . . . . . . . . 74
B.2 Take Home Survey Responses . . . . . . . . . . . . . . . . . . . . 83

List of Figures
1 Feng Shui Theory based on Pa-Kua Image, photo on fenshuiart.com[6] 11
2 Rosina Asser-Pardo who used to hidden during the occupation
was being interviewed for project “Memories of Occupation in
Greece”, photo by Dimitris Polidoropoulos in 2016[7] . . . . . . . 12
3 A screenshot from “I’M A MAN” demo, reproduced by the author
from Derek Ham’s virtual reality project[8] . . . . . . . . . . . . 14
4 A screenshot from ”GoFind!” demo, reproduced by the author
from Loris Sauter’s augmented reality project[?] . . . . . . . . . 15
5 “reconstructive digital modeling and AR application on the cham-
ber of the First Italian Parliament located in the courtyard of
Palazzo Carignano in Turin”, photo reproduced by the author
from Fabrizio Natta and Michele Ambrosio’s augmented reality
project[9] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6 The display of a church based on location-based augmented re-
ality, reproduced by the author from R˘azvan Gabriel Boboc’s
augmented reality work[1] . . . . . . . . . . . . . . . . . . . . . . 17
7 The method structure (Made by the author) . . . . . . . . . . . . 18
8 Location of the Hill District neighborhoods within the City of
Pittsburgh, photo from Wikipedia ”Hill District”[10] . . . . . . . 19

3
9 ”Shake dancer, possibly “Gypsy Rose Lee” performing with band,
including Tommy Turrentine on trumpet, at Bambola Social Club”,
photo by Charles “Teenie” Harris, c. 1938-1946[11] . . . . . . . . 21
10 Selected historical sites in this research, made by the author . . . 23
11 “Pharmacist William Goode, standing outside of Goode Phar-
macy with advertisement for Nucoferrin in window, Wylie and
Fullerton Avenues, Hill District”, photo by Charles “Teenie” Har-
ris, c. 1950-1965[12] . . . . . . . . . . . . . . . . . . . . . . . . . 23
12 ”Interior of Musician’s Club, Local 471, with striped awning over
bar, television in corner, and round bar stools”, photo by Charles
“Teenie” Harris, c. 1955-1975[13] . . . . . . . . . . . . . . . . . . 24
13 ”Exterior of Rhumba Theatre seen from across street, Fullerton
Street, Hill District”, photo by Charles “Teenie” Harris in 1962[14] 25
14 ”Demolition of the Bethel A.M.E Church began in 1957” © Rev.
Dr. Dale Snyder[15] . . . . . . . . . . . . . . . . . . . . . . . . . 26
15 “The construction of the Civic Arena in the Lower Hill district.
Parts of Wylie Avenue, Crawford Street, Fullerton Avenue and
Webster Avenue were demolished to make room for the large
silver dome. These streets were significant business and cultural
centers of the Hill District prior to their demolition. Construction
of the arena began in 1958”, photo by Charles “Teenie” Harris in
1958[16] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
16 Freedom Corner in Pittsburgh (Photo by the author) . . . . . . . 28
17 All photo collections are manually tagged and categorized based
on time and theme. Each square in Figure represents 10 photos
(Made by the author) . . . . . . . . . . . . . . . . . . . . . . . . 29
18 Images from Pittsburgh City Photographer (PCP) Collection and
Teenie Harris Photography Collection about Hill District[17][18],
organized by the author . . . . . . . . . . . . . . . . . . . . . . . 30
19 Sala Udin’s interview about the Hill District community of Pitts-
burgh, reproduced by the author from “TheHistoryMakers” Oral
History Collection website[19] . . . . . . . . . . . . . . . . . . . . 31
20 The BBC Sound Effect data set, reproduced by the author from
”BBC Sound Effects” website[20] . . . . . . . . . . . . . . . . . . 35
21 the playlist of “La Place”, reproduced by the author from All-
Music website[21] . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
22 Adobe Aero interface (Made by the author) . . . . . . . . . . . . 37
23 Pretest demo screenshots to display local economic activities,
the urban renewal program and civic activities around the Free-
dom Corner (Made by the author), with materials from historical
photo collections[17][18] . . . . . . . . . . . . . . . . . . . . . . . 38
24 All constructions within the project (Made by the author) . . . . 39
25 Model building based on historical map (Made by the author), the
map which is used as a reference is a historical map of Pittsburgh
in 1923[22] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4
26 Model building based on historical photos (Made by the author),
the historical photos used as references (right) are from historical
photo collections, photo by Charles “Teenie” Harris[17][18] . . . . 40
27 Texturing process with textures from historical photo (Made by
the author) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
28 Comparison between historical images and model assets (Made
by the author) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
29 The Armature Attachment process of human assets (Made by the
author) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
30 2D assets used in the augmented reality scene (Edited by the
author), with materials from historical photo collections[17][18] . 43
31 The inspector setting screenshots for location setting in Unity
(Made by the author) . . . . . . . . . . . . . . . . . . . . . . . . 45
32 Getting latitude and longitude data of geography location from
Google Map (Made by the author) . . . . . . . . . . . . . . . . . 46
33 The audio clips import settings in Unity (Made by the author) . 47
34 The spatializer plugin install process (Made by the author) . . . 48
35 Parameter settings of “AudioSource” in Unity (Made by the author) 49
36 Screenshots of the interface with the mini-map included (Made
by the author) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
37 Asset Distribution (Made by the author) . . . . . . . . . . . . . . 53
38 On-site augmented reality scene of Rhumba Theater (Made by
the author) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
39 The framework of user flow in the augmented reality walk (Made
by the author) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
40 A participant was viewing the virtual billboard and people in the
protests on the historical site(Photoed by the author) . . . . . . 57
41 A participant was taking the interview after the augmented real-
ity walk (Photo by the author) . . . . . . . . . . . . . . . . . . . 59
42 Form of assets information, visual part . . . . . . . . . . . . . . . 75
43 Form of assets information, auditory part . . . . . . . . . . . . . 76

5
1 Introduction
According to a Post-Gazette research, 299 out of 1889 historically significant lo-
cations in Pittsburgh were lost, or about 16 percent of the individual structures,
and that many more were expected to be destroyed in the near future[23]. The
loss of structures that are on the register is only a small portion of the overall loss
of the city’s historical environment. This research indicates that in Pittsburgh,
the use of urban space resources has increased in intensity with the growth of
urban settings and the commercial regeneration of cities and more and more
register-listed buildings are being demolished to make room for potential future
expansion as a result of the construction of commercial structures and public
services. The loss of historic assets and communities has been hastened by the
lack of an efficient program for the preservation of historic buildings and invest-
ment in upkeep. Intangible heritages including traditions, memories, and the
distinctive qualities of the neighborhoods have also vanished in addition to the
actual buildings and structures. Urban development ought to be a dynamic,
ongoing process with many overlapping layers, including those of historical sig-
nificance. There is an urgent need to analyze and resolve the issue of how to
protect intangible culture and its carriers during the urban regeneration process.

The heritage itself is more than simply an object on the site since it is both
a monument and a component of the living fabric. It also includes immaterial
elements like local cultural practices, traditions, and recollections. Scholar of
Roman architecture and urban history Gustavo Giovannoni, who is credited
with coining the phrase ”urban heritage,” emphasized the value of protecting
historical sites while also recognizing the value of urban growth[24]. Based on
the notion of contemplating heritage protection on a broad and inclusive scale,
a landscaping approach to urban heritage management developed at the turn
of the twenty-first century, which supported a strategy for thoroughly mapping
and gathering the city’s natural, cultural, and human resources as well as in-
cluding the public and stakeholders in dialogue. A comprehensive contextual
view of urban heritage that incorporates the idea of landscape resulted from
the expansion of the concept of cultural heritage to include associative values
and varied viewpoints from many participants instead of a focus on tangible
assets.[25] In order to safeguard historical assets, the public ”should be called
on for recommendations and aid,” according to the United Nations Educational,
Scientific, and Cultural Organization (UNESCO)[26]. The historic urban land-
scape (HUL) is a tool for managing cultural heritage that is based on the idea
that “when an urban settlement is properly managed, initiatives, opportuni-
ties, and development can contribute to both quality of life and conservation
of cultural heritage, while ensuring a social diversity and justness”[27]. It sug-
gests that it is important to put emphasis on the recognition and protection
of heritage, which represents the intangible component, and public involvement
in urban developing process. However, in light of the grave concern over the
clash between demands for urban growth and historic preservation[28], it is ur-

6
gently necessary to develop tools using new technologies and media in order to
portray heritage and foster a more equitable dialogue among many stakeholders.

Intangible heritage values, such as oral traditions, traditional arts, commu-

nity engagements, celebratory events, can be seen as a collection of “traditions
and living expressions inherited from our ancestors and passed on to our de-
scendants” as Jigyasu points out in his research[29]. The transmission of these
beliefs from one generation to the next is crucial for the long-term growth of
our culture and society. In addition, a deeper knowledge of intangible values
influences how people see the utilization of urban resources beyond economic
considerations, which offers the potential to address urban development con-
cerns in a more deliberate and thorough manner.

It has been a number of decades since museums, historic sites, and other
relevant organizations used digital technology for legacy conservation, demon-
strating the possibility of merging tangible and intangible components of his-
torical structures and items through digitization. Digitization specifically offers
chances for reviving previously suppressed narratives at the location[30][31].
More specifically, in order to advance social justice, it is crucial to pay attention
to marginalized groups and their archives, according to a probable forgotten
narrative in the process of urban development. To reflect certain distinctive
or unique viewpoints that are easily neglected in a more broad historical doc-
umentation, these archives might be constructed based on historical resources
like oral history recordings from tiny communities or groups. As an illustration,
consider this oral history project by Densho, a nonprofit organization founded
in 1996, which details the Japanese American experience from immigration to
reparations with a special emphasis on the mass imprisonment during World
War II.[32]
Augmented reality creates an interactive environment by modifying the phys-
ical world based on computer-generated perceptual data. The user’s perspective
of the real environment is merged with the digital world, which improves the
integration of the two settings. In addition, augmented reality frees designers
and producers from the institutional processes that are typical of traditional
exhibit mediums for cultural assets[33]. Historical items can be found and posi-
tioned on the actual location inside metropolitan areas. This portrayal is more
vivid and comprehensive due to the spatial continuity of history. Additionally,
augmented reality makes it simple to offer individualized and personalized pre-
sentations of content. If they so want, anybody can create their own personal
exhibits. Moreover, neither the general public nor community members can
have an intuitive or immersive experience when learning about the intangible
cultural heritage because the bulk of the aforementioned materials are cut off
from the real place and physical space of the legacy. Because of this, augmented
reality may be utilized to present local populations’ memories on historical lo-
cations, illuminating the local history of community.

7
1.1 Research Question
This study investigates how augmented reality might produce a location-based
hybrid urban environment at the intersection of urban heritage and community
assets. More specifically,

a) How might location based augmented reality help people see the intangi-
ble aspects of urban heritage?
b) How might hybrid experiences improve understanding of local intangible
heritage?

I offer ”Hybrid View of Lost Heritage in Lower Hill District”, an augmented

reality tour at Pittsburgh’s historical monument Freedom Corner and its neigh-
borhood, to address the concerns raised above. This augmented reality tour
told tales of how urban renewal in the 1950s influenced the present and future
of this historic district and the growth and collapse of nearby communities. The
semi-virtual contents immerse individuals in historical events using oral history
data gathered and historical visual materials from public repositories. People
may study the historical legacy and have a greater grasp of the mutual mapping
of history and reality through the time series focused and location based ex-
perience. By utilizing this overlay knowledge, the intangible aspect of regional
tradition is once more made concrete.

1.2 Contribution
By drawing from the previous works about digital heritage, this work adds to
the field in the following ways:

a) Create a location-based augmented reality walk to explore, learn about,

and comprehend the social, cultural, and historical facets of intangible heritage
or destroyed tangible heritage. Investigate and comprehend how participants
interacted with the walk’s auditory and visual components based on how they
responded to it.
b) Showcase the design process for a location-based augmented reality ap-
plication for heritage exhibition. Collection of historical assets, system develop-
ment, and experience design are all covered. Researchers with little expertise
may create a location-based heritage walk more easily based on this procedure,
which examines the presentation techniques of various aspects of local history.
c) Use the Lower Hill District and the neighborhood near Pittsburgh’s Free-
dom Corner as an example to study how the augmented reality project con-
tributes to the understanding of local history and heritage.
d) Provide a review of recent literature in the research field about digital
heritage

8
1.3 Thesis Structure
In the thesis, I first present the research review about heritage definition and its
protect. Then I go through the field of work about using extended reality for her-
itage study and protection. Also I offer a case study of Pittsburgh’s Lower Hill
neighborhood and some specific historical spots covered in the research.(chapter
two) After that, I provide a hypothesis for the heritage preservation based on
location-based augmented reality technique and its contributions.(chapter three)
After that, I will describe the workflow of content gathering, system develop-
ment, and experience design for the design development of ”Hybrid View of Lost
Heritage in Lower Hill District” (chapter four). The analysis of participant re-
sponses from the user test will be used to deliver the experience assessment
result in chapter five. The conclusion (chapter six) and future work (chapter
seven) will be covered in the last sections.

9
2 Background Research
2.1 Intangible Cultural Heritage
2.1.1 Definition and Value Evaluation of Intangible Heritage
Heritage is what we have preserved from the past, a record of the present, and
a valuable asset to hand down to the next generation. It may take on a vari-
ety of forms, including tangible ones like places, structures, and landscapes as
well as intangible ones such as oral histories, performance arts, social customs,
religious observances, holiday celebrations, skills and customs relating to envi-
ronment and the cosmos, as well as awareness and understanding relating to
traditional handicrafts. Even though intangible heritage is more vulnerable and
invisible compared to physical heritage, it’s still an essential aspect in preserving
cultural variety[34]. Based on the evaluation of Su, Li and Kang’s study, the
value of intangible cultural heritage can be divided into four aspects: historical
value, scientific value, aesthetic value and economic value[35].

Historical value refers to how well intangible cultural heritage captures the
experiences of locals during a particular era. In the study of Foster and Gilman
in 2015, the word ”local” puts the focus on particular locations which contrasts
it with the word ”global’ ’via the gathering, comprehension, and analysis of
resources and materials from the community[36]. Intangible cultural heritage
therefore encourages a more accurate, thorough, and impartial view of history
and can be seen as a flaw in official historical documentation. Secondly, intangi-
ble cultural heritage also contains scientific values which means local knowledge,
ideas, and techniques may be gathered and established in this approach[37].
Thirdly, the aesthetic value of intangible cultural heritage supports the idea
that plays, performances, tales, and other intangible arts may enhance people’s
spirituality[38]. Finally according to economic values, the intangible cultural
heritage might be used as a resource to meet the needs of the local population
by promoting tourism or the rest of the economy. In this condition, heritage
sites are more valuable when there are more human activities on the site, and
conservation that encourages retrofitting of assets is more valuable than inactive
safeguarding[39].

Consider the Chinese practice of ”Feng Shui”[40]. As a practical art that is

deeply ingrained in human life, it depicts people’s social and cultural activities
over the course of several historical eras[41]. For both domestic space design
and cosmopolitan planning and design, “Feng Shui” has been used as a guide.
It is also regarded as a philosophical theory on how people and nature interact.
By utilizing the idea in the areas of communication, design, and tourism, it also
adds economic value. For instance, when building classical Chinese gardens
such as the Summer Palace in Beijing, Feng Shui was used as an important
consideration in determining building orientation and vegetation design.[42] It
is hard to completely comprehend or examine the design of the Summer Palace

10
Figure 1: Feng Shui Theory based on Pa-Kua Image, photo on fenshuiart.com[6]

without knowledge of this intangible legacy.

Oral transmission of intangible heritage is possible within a community.

Even if there are some distinctions between oral history and intangible cultural
heritage: oral history aims to gather and preserve historical data from individ-
uals and organizations whereas intangible cultural heritage strives to maintain
cultural heritage with community members by safeguarding the mechanisms
that allow traditions and common knowledge to be transmitted, oral narratives
could be a valuable tool for comprehending historical occurrences or study-
ing cultural heritage.[43] Take the project called “Memories of Occupation in
Greece”[7] as an example. Multimedia testimonials about the German occu-
pation of Greece from 1940 to 1944 were gathered and archived as part of the
project. Throughout all, 93 interviews with survivors of the time were un-
dertaken in Greece including different kinds of victims of this historical event

11
like individuals who were held in detention camps or those who saw reprisals.
Generally, the research prompts thoughts on the function of cultural heritage
in evoking memories and the possibilities of taking oral history as a source for
study. Therefore, it can be claimed that compiling and analyzing these life expe-
riences may be considered as a way to strengthen and protect a group’s identity.

Figure 2: Rosina Asser-Pardo who used to hidden during the occupation was
being interviewed for project “Memories of Occupation in Greece”, photo by
Dimitris Polidoropoulos in 2016[7]

2.1.2 Intangible Heritage protection

Heritage draws attention to values and objectives within the community by as-
sisting in the formation of local identity. Both directly and indirectly, heritage
has an impact on culture and society. The cultural legacy, which preserves
evidence of historical, social, economic, and civic elements of community as-
sets, is eliminated from urban space owing to accidents, natural disasters, or
unrealistic urban development plans. For instance, in the research about the
reconstruction after a earthquake in Irpinia, Moscaritolo claimed that the work
progress shouldn’t just focus on the repair of tangible “objects” like infrastruc-
tures or houses, but should also consider about the rebuilding as a “ complex
social process” that takes cultural contexts, expectations, and thoughts about
the next generation into consideration.[44] This exposed the importance of in-
tangible cultural heritage protection for the local community and the whole
cultural background.

Many scholars have previously concentrated on the digital reconstruction of

12
historically significant sites in the field of cultural heritage preservation meth-
ods. A previous approach for heritage revival in the digital age has been created
to address the issue of missing data and is based on crowd-sourcing and SfM
algorithms.[45][46] Buildings and even massive structures are rapidly becoming
part of the restoration scope. Although this strategy has numerous advantages
for preserving and rebuilding cultural legacy, heritage is more faithfully restored
as an item or physical asset. The physical form of the legacy is not combined
together with other, more intangible elements, such as local culture, public ac-
tivities, economic developments, etc. This research, on the other hand, offers
a strategy for re-creating intangible heritage through an engaging story that is
delivered in context.

Intangible cultural heritage also changes over time. Actually, it constantly

innovates and changes because it is firmly established in specific areas and neigh-
borhoods and is updated as those communities grow. Therefore, in order to
preserve it as ”a source of invention and innovation,” the preservation strat-
egy for intangible cultural assets should also take durability and renew-ability
into account.[47] Many earlier academics have cited community involvement as
a key strategy for protecting intangible cultural resources. In a 2013 study by
Bakar, the authors discussed a conservation strategy based on the statistical
correlations between social and demographic variables and the degree of com-
munity engagement in intangible cultural resources. It found that community
issues like “acculturation and deculturation, preference of popular culture, weak
family institution” affects the involvement of local intangible cultural heritage
protection.[48]

Previous studies indicate a high level of community involvement is essential

for heritage protection and community identification.[49] But the conventional
recording of heritages shown as historical markers, texts, videos, or photographic
archives lacks audience participation and interaction, which makes it tougher to
pique the public’s attention. Additionally, because the majority of the aforemen-
tioned resources are disconnected from the actual location and physical space
of the legacy, neither the general public nor community members can have an
intuitive or immersive experience while learning about the intangible cultural
heritage.

2.2 Extended Reality for Intangible Heritage Protection

Virtual reality and augmented reality have already been used by a lot of scholars
and artists to depict or portray both tangible and intangible heritages, such as
historical structures, social events, local living situations, etc.[50][51][52] These
media give a chance to dynamically and interactively convey assets of urban
history by more accurately displaying site information and simulating the set-
ting through visual and auditory components. By transporting users back in
time and making them ”eyewitnesses” to the historic figure, many virtual real-

13
ity projects that generate immersive experiences concerning local heritage are
successful in fostering an intimate attachment between people and the past. For
instance, independent virtual reality developer Derek Ham’s project called ”I’m
a man” brings audiences back to the memory about the death of Martin Luther
King Jr.[8] However, the entirely virtual experience is disconnected from the
physical locations since viewers are more likely to be confused by the spatial
ambiguity and lose track of the understanding of history and cultural heritage
based on a specific site. Furthermore, since a virtual reality experience requires
both touch devices and headgear equipment, the usage settings for this tech-
nology tend to be interior and static. When researchers attempt to integrate
digital assets with physical surroundings, both historical and contemporary, it
introduces additional limits.

Figure 3: A screenshot from “I’M A MAN” demo, reproduced by the author

from Derek Ham’s virtual reality project[8]

As a result, several academics concentrated on augmented reality’s uses in

the field of heritage protection. Nowadays, businesses and organizations are us-
ing augmented reality applications for heritage conservation, tour services, and
public records due to technological improvements and the widespread use of mo-
bile devices. As a virtual overlay over a real location, augmented reality might
potentially enhance exhibiting approaches as well as the user experience. This
is in contrast to other types of extended reality techniques like virtual reality
and mixed reality. For instance, Sauter presented an location-based historical
photo querying application called ”GoFind!” in 2018. This project allows users
to view multimedia collections based on searching for specific site, which makes
the heritage assets more accessible and understandable.[53]

14
Figure 4: A screenshot from ”GoFind!” demo, reproduced by the author from
Loris Sauter’s augmented reality project[?]

The two types of augmented reality are marker-based and location-based.

Mark-based augmented reality, sometimes referred to as image recognition aug-
mented reality, needs a marker to start the interaction, typically a trigger picture
or QR code. Because it’s simple to create and easy to use, this type of program
is appropriate for advertising and entertainment usage to boost engagement.[54]
For instance, in Natta’s project in 2022, a marker-based augmented reality ap-
plication was introduced to make a reconstruction of a parliament in Turin as
shown in Figure 5.[9] However, the flaw is more blatant: marker-based aug-
mented reality is less location-agnostic because it only functions when the cam-
era is close to the marker. In large metropolitan areas that need user mobility
and visual alteration like urban environment, marker-based augmented reality
is not adaptive enough.

The technology behind location-based augmented reality, in contrast to

marker-based augmented reality, doesn’t need any actual markers or labels to
function. Recent studies look at ways to make historical audiovisual collec-
tions available on mobile devices, allowing for the layering of historical informa-
tion with present-day views via location-based augmented reality. For example,
Boboc presented a project which uses location-based augmented reality to place
the visual elements of an architecture on the according historical spot.[1] Pro-
grams like this are able to browse the back end for all results and save their
locations. Users may move around the city environment, and when they arrive
at a spot with historical background, they will receive a signal. Users are allowed
to engage with concrete elements of urban heritage in a more approachable and
engaging way thanks to the combination of digital and physical situations. The
display of dispersed and static goods, however, also has drawbacks. Although
it may be possible to represent physical heritage through static objects, there

15
Figure 5: “reconstructive digital modeling and AR application on the chamber
of the First Italian Parliament located in the courtyard of Palazzo Carignano
in Turin”, photo reproduced by the author from Fabrizio Natta and Michele
Ambrosio’s augmented reality project[9]

are yet no workable display or narrative options for intangible heritage like hu-
man activities or historical events. On the other hand, through a multimedia
narrative experience in urban areas, this article truly offers a way to showcase
intangible heritage via location-based augmented reality.

16
Figure 6: The display of a church based on location-based augmented reality,
reproduced by the author from R˘azvan Gabriel Boboc’s augmented reality
work[1]

17
3 Hypothesis
Based on the aforementioned analysis and research, I predict that location-
based augmented reality could offer a possibility to display intangible cultural
heritage in a way that is more immersive and intuitive. Building a location-
based augmented reality project to combine multimedia materials and map them
with actual places is easier compared to traditional documentary approaches
like photograph gathering, oral history collection, or documentary film. The
augmented reality walk’s blend of visual, audio, and interactive components will
also provide a more engaging experience and aid users in developing a greater
knowledge of the region’s intangible cultural heritage.

4 Method Statement
In this chapter I discuss the development of “Hybrid View of Lost Heritage in
Lower Hill District”, the location-based augmented reality walk, from the fol-
lowing aspects: content collecting, system building and experience design.

Figure 7: The method structure (Made by the author)

4.1 Case Study: Lower Hill District

4.1.1 History and Community Assets of the Hill District
One of Pittsburgh’s original communities, the Hill District, was established in
the early 1800s. The Lower Hill, the Middle Hill, and the Upper Hill, collec-
tively known as the ”Sugartop”, are three communities that are situated on the
mountains above downtown Pittsburgh. The Lower Hill District neighborhood
was situated between Centre and Bedford Avenues. There used to be a vibrant

18
Figure 8: Location of the Hill District neighborhoods within the City of Pitts-
burgh, photo from Wikipedia ”Hill District”[10]

living area with diners, barbershops, cinemas, grocery stores, schools, churches,
community centers, and bars here. It’s described as “a city within a city” as
well as “Crossroads of the World”.[55] This neighborhood intrigues me not only
because of its extensive and rich history, but also because the Lower Hill locals
were forced to leave the area and the entire living neighborhood was torn down
during the urban development process. All of the bearers of intangible heritage
also vanished. It’s difficult to trace the history of a location when all that re-
mains are the physical assets there, and a parking lot currently occupies the
historical site. I concluded that it may be a great case study for figuring out
how to rebuild the intangible heritage on the site using a hybrid representation
made using augmented reality.

Hill District used to be a center of the abolitionist movement in Pittsburgh.

150 liberated slaves from Virginia arrived in Pittsburgh during the American
Revolution. They were among the first people to settle in Hill, establishing en-
terprises and the Bethel A.M.E. church in 1818.[56] The Hill served as a hub for
the underground activities and the abolitionist movement before the American
Civil War under the direction of Martin Delany, Reverend Lewis Woodson, and

19
John Vashon.[57]

The Hill District contained 25 different races, according to the 1940 Census.[58]
Many people from different racial groups like Jews, Italians, Greeks, and African
Americans were involved in local business, regional activities and amusement
establishments which gave the Hill’s residents a thriving community life.

During this time, the local community’s cultural, social, and economic com-
ponents were lively and vibrant. The Monticello-Delaney Rifles basketball
team, which won the Colored Basketball World Championship in 1913, was
one of the successful African American basketball and baseball teams in the
Hill District.[59] August Wilson, a Pulitzer Prize and Tony Award winning
writer who was born and raised in the Hill District, wrote 10 plays on lives
in the African-American community.[60] From the early 1920s until the 1960s,
jazz developed and flourished in the bars, nightclubs, and playhouses of the Hill
District. In the past, one of Pittsburgh’s original jazz clubs was housed in the
structure at 1213 Wyle Avenue called Local’s 471 Musician’s Club. For thirty
years, this club featured outstanding jazz performances by both local and for-
eign performers and it also became an important place to offer all the musicians
brotherhoods and support. At the club, they chatted, rehearsed, enjoyed music
and also, delivered jazz performances.[61] The Bambola Social Club, which be-
came a venue for performances by singers, shake dancers and bands, was one of
Pittsburgh’s Hill District’s most well-known clubs during the 1940s.[62]

4.1.2 Urban Renewal Project in the Lower Hill

Urban Renewal, a movement for urban regeneration, existed from 1955 until
1963. With this project, Pittsburgh’s ”slum” that was near to the city center
would be eliminated. It viewed the Hill District, which was largely made up of
African Americans, as a ”blighted barrier” to the development of the city. As a
result, Mayor Lawrence and his administration opted to build the Civic Arena in
the Lower Hill District in order to uplift this ”slum” neighborhood. 1,239 black
households were evicted as a result of the Urban Redevelopment Authority’s
destruction.[63] From 17,334 people in 1950 to 2,459 people in 1990, the Lower
Hill lost lots of population during this period. People who were compelled
to leave the region typically settled in areas that were racially homogeneous,
which made the city’s segregation issue worse. Pittsburgh was one of the most
segregated American major cities by 1960.[64] Additionally, some historically
significant buildings were removed, including the Bethel A.M.E. Church, whose
members are still suing for damages currently.[15] Following the demolition of
the Lower Hill District, public structures like the Civic Arena were constructed.
However, in 2012, the Civic Arena was also taken down from the location.[65]

20
Figure 9: ”Shake dancer, possibly “Gypsy Rose Lee” performing with band,
including Tommy Turrentine on trumpet, at Bambola Social Club”, photo by
Charles “Teenie” Harris, c. 1938-1946[11]

21
4.1.3 Civic Right Activities around Freedom Corner
This landmark commemorates a pivotal moment in Pittsburgh’s vast Civil
Rights history and is situated at the junction of Centre Avenue and Craw-
ford Street in the city’s historic Hill District. Numerous longtime residents were
evicted due to the Lower Hill District’s Urban Renewal initiative. A billboard
erected at Crawford Street and Centre Avenue following the URA’s removal
of 8,000 people, 413 companies, and 1,300 structures from the Hill. It said,
”Attention: City Hall and the URA. NO Redevelopment Beyond This Point.”
From then, a succession of nonviolent demonstrations took place at this loca-
tion, which later came to be known as Freedom Corner. An important turning
point occurred in 1963 when more than 2,000 Pittsburghers left Freedom Corner
to see D. Martin Luther King Jr. delivered his renowned ”I Have A Dream”
address in Washington, DC.[66] Local communities can congregate at Freedom
Corner to express their demands for the protection of their own rights.

4.1.4 Heritage Site Selection and Evaluation

This section will discuss the site conditions at Freedom Corner and the Lower
Hill District. To create a site-specific hybrid experience in a long-gone neighbor-
hood, it is essential to have a firm understanding of the physicality and spatiality
of the place both in the past and in the present. Before the 1960s, sites includ-
ing Goode’s Pharmacy, Local 471’s hall and Musician’s Club, Rhumba Theater,
and Bethel African Methodist Episcopal Church served as significant economic,
social, or cultural centers. The development of Civic Arena, was made possible
by the demolition of all the aforementioned sites during the urban renewal. Lo-
cals conducted demonstrations to oppose more construction in the Hill District
and to demand affordable housing, establishing the Freedom Corner as a forum
for Pittsburghers to voice their opinions and needs.

Site 1: Goode’s Pharmacy (1303 Wylie Avenue)

1303 Wylie Avenue used to be the address of Goode’s Pharmacy. The ”cultural
and commercial heartbeat” of the Hill District was where Fullerton Street and
Wylie Avenue intersect. In 1949, William Goode launched his second pharmacy
shop, which became well-known in the neighborhood as an example of a local
black-owned company. The Goode’s Pharmacy had a fantastic soda fountain,
which made it the ideal location for young people to congregate and mingle. In
order to serve the neighborhood, it was open 24/7 and provided free delivery.[67]

Young African American pharmacists were unable to receive training at

pharmacies managed by white people at the time due to racism; as a result,
Goode’s Pharmacy was viewed as a safe haven for them to grow in their profes-
sion. According to William Goode’s daughter, her father believed that “it was
the community that had given him his livelihood” and “helping these young
men and women to achieve their goals was his way of giving back”.[68] Mr.
Goode’s family and friends established the scholarship in honor of his lifelong

22
 Figure 10: Selected historical sites in this research, made by the author

commitment to assisting worthy African-American students in achieving their

goals.

Figure 11: “Pharmacist William Goode, standing outside of Goode Pharmacy

with advertisement for Nucoferrin in window, Wylie and Fullerton Avenues, Hill
District”, photo by Charles “Teenie” Harris, c. 1950-1965[12]

23
 Site 2: Local 471’s hall and Musician’s Club (1213 Wylie Avenue)
The Hill District’s Musician’s Club of Musicians Union Local 471 was situated
at 1213 Wylie Avenue. It functioned as a hub for black and white Pittsburgh
artists to interact, converse, and play jazz music. A trumpet player who is a
member of Local 471 named Charles Austin said the club’s openness allows mu-
sicians to perform, create bands, and even exchange advice.[69]

Figure 12: ”Interior of Musician’s Club, Local 471, with striped awning over bar,
television in corner, and round bar stools”, photo by Charles “Teenie” Harris,
c. 1955-1975[13]

Site 3: Rhumba Theater (53 Fullerton Street)

Red Hot Theater Located in the Hill District of Pittsburgh, on Fullerton Street
just off Wylie Avenue. There were at least four names for the theater. In 1926,
it adopted the name Burke’s Theater. It had taken on the name Golden Theater
by 1934. It was changed on September 25, 1935, into Javo’s Jungle, a nightclub
that operated until February 1937. It was renamed Rhumba Theater in April
1937. Jacob Solz oversaw the theater, which catered to the African-American
neighborhood. In 1957, it was still in use. In 1958, it was taken apart to make
place for the Civic Arena.

Site 4: Bethel African Methodist Episcopal Church

After a large fire completely destroyed the former Bethel Church in Pittsburgh’s
central business district, the Bethel A.M.E. Church at Wylie Avenue and Elm

24
Figure 13: ”Exterior of Rhumba Theatre seen from across street, Fullerton
Street, Hill District”, photo by Charles “Teenie” Harris in 1962[14]

Street was constructed in 1906. The first ”colored” religious organization in

Pittsburgh was Bethel A.M.E., which was established on Front Street in 1808
as the ”African Church.”[56] As a crucial center for intellectual development and
independence-related activities in the 19Th century, Bethel A.M.E. acted as a
buffer for African Americans’ social standing.[70] The Urban Renewal Project
led to the demolition of the chapel in 1957. Actually, it was the final piece of
architecture around Wylie Street to be taken down. The 2010-built PPG Paints
Arena for the Penguins uses the site of Bethel A.M.E. as a parking lot.[71]

Site 5: Civic Arena

Between Pittsburgh’s Lower Hill neighborhood and downtown, Civic Arena was
constructed in 1961 as a result of the Urban Renewal Project within the 1950s.
It was initially built to house the Civic Light Opera, and it has since held a
variety of family events as well as performances by musical greats including
Frank Sinatra, The Rolling Stones, Garth Brooks, and The Grateful Dead. It
was the first sizable sports arena of its sort in the world and included a 170,000
square foot retractable roof.[72] Civic Arena, however, was shut down in 2010
and destroyed in 2012 because City Planning and City Council were unwilling
to accept the proposal for this building’s historic status.[73]

Site 6: Freedom Corner

25
Figure 14: ”Demolition of the Bethel A.M.E Church began in 1957” © Rev.
Dr. Dale Snyder[15]

In order to defend their own rights, locals frequently stage demonstrations at

Freedom Corner. A billboard arose near Crawford Street and Centre Avenue
to protest the government’s intention to relocate urban redevelopment from the
Lower Hill District to the Middle and Upper Hill. After that, this location
is used for public events or demonstrations. In Pittsburgh, it benefits all the
neighborhoods, not just the African American ones.

26
Figure 15: “The construction of the Civic Arena in the Lower Hill district. Parts
of Wylie Avenue, Crawford Street, Fullerton Avenue and Webster Avenue were
demolished to make room for the large silver dome. These streets were signifi-
cant business and cultural centers of the Hill District prior to their demolition.
Construction of the arena began in 1958”, photo by Charles “Teenie” Harris in
1958[16]

4.2 Content Collecting

4.2.1 Historical Images
The image contents are collected from two primary sources:

a)Pittsburgh City Photographer (PCP) Collection

The Pittsburgh City Photographer (PCP), a unit of the Pittsburgh Public
Works Department, was created in 1906 and was assigned with snapping pho-
tographs of the city’s infrastructure. Public spaces like parks, streets, build-
ings might require photography due to the requirement of government agencies.
PCP photographers also kept track of the locations of the images. This re-
search selected on 716 photos in the Pittsburgh City Photographer Collection
on ”Historical-Pittsburgh” website about the Hill District.[17]

b)Teenie Harris Photography Collection

Photos in this collection were taken by Pittsburgh photographer Charles Teenie
Harris who depict black lives in Pittsburgh between 1920s and 1970s. Different

27
 Figure 16: Freedom Corner in Pittsburgh (Photo by the author)

aspects of human daily activities like Marriage, deaths, portraits, religious ser-
vices and other events happening within urban landscapes were displayed with
photos. This research chose 83 photographs in the collection about the Hill
District.[18]

All photos from above two resources are manually tagged and categorized
according to time and theme as shown in Figure 16. Each square in Figure
represents 10 photos. Examples of photo collections are shown in Figure 17.

4.2.2 Online Oral History Collection

Oral history documentation is also an important resource in this research to pro-
vide evidence for digital modeling and scenario construction. There are three
main resources I leveraged in this project. The first is TheHistoryMakers oral
history archive which consists of interviews with previous local residents in the
Lower Hill District.[74] The second one is African American Jazz Preservation
Society of Pittsburgh Oral History Project which contains the memory about
Jazz bands and Local 471 musician club history.[75]

The third resource is documentaries about the Hill District including 3 spe-

28
Figure 17: All photo collections are manually tagged and categorized based on
time and theme. Each square in Figure represents 10 photos (Made by the
author)

cific videos. The first one is a YouTube video provided by Pittsburgh HillDistrict
named The Renewal of Pittsburgh’s Hill District created by Soleil Nagoda and
Laurel Cooper which contains several interviews with local residents and their
experience about the Urban Renewal project.[76] The second video is a doc-
umentary called Civic Arena Engineering and Innovated in Pittsburgh made
by Rick Sebak and WQED which is about the construction of Civic Arena in
Pittsburgh. Several interviews with engineers and designers are included in this
video.[77] The third one is Pittsburgh People and Places Tour: Stop 4 which
introduces the history of the Freedom Corner in Pittsburgh.[78]

4.2.2.1 “TheHistoryMakers” Oral History Collection

By collecting and documenting oral history from “TheHistoryMakers” which
is the largest African American video oral history public archive. Based on
how well they related to the topic of memories of living in the Hill District, I
mainly chose two interviews from the collection (Sala Udin and Bev Smith).
The database of ”TheHistoryMakers” contains transcripts of every interview,
making it simple to look for the necessary information. The appendix contains
a list of all the audios utilized in this study. Some examples are displayed as
follows.

29
Figure 18: Images from Pittsburgh City Photographer (PCP) Collection and
Teenie Harris Photography Collection about Hill District[17][18], organized by
the author

a)Sala Udin’s Interview

Sala Udin was a city administrator and civil rights activist. He was born on
February 20, 1943 in Pittsburgh. He helped to establish Black Horizons Theatre
in 1967, a company that produced plays by black authors to embrace Black
Power. Additionally, he held the position of Sixth District Councilman. Udin
talked about his early years in the Lower Hill District in this interview, including
his memories of Goode’s Pharmacy, the Rhumba Theatre, his mother’s sandwich
store, etc. He also brought up the process of Pittsburgh’s Urban Renewal and
how many buildings, including the Bethel A.M.E. Church, were demolished.[79]

Sala Udin’s Interview Clip Examples:

“...Fullerton and Wylie was the heartbeat–both the cultural and commercial,
there were stores, newsstands, cigar stores, grocery stores, drug stores. The
Goode family owned the drug store. Everybody knew Goode’s Drug Store [Goode’s

30
Figure 19: Sala Udin’s interview about the Hill District community of Pitts-
burgh, reproduced by the author from “TheHistoryMakers” Oral History Col-
lection website[19]

Pharmacy]. I thought all drug stores were called Goode’s Drug Store... There
were a lot of Jews there because the Jews tended to dominate the merchant group
that owned stores and bars and places in the Hill District, and some of them lived
there. But it also included a lot of Italians, Syrians, and Lebanese. Those are
the groups that I remember, and a lot of blacks...I would say that 30, 40, maybe
50 percent of the businesses were black-owned. The doctors and dentists had
their offices in the community. The–several of the bars were owned by blacks.
Pat’s Place was a newsstand and cigar shop that was black. Goode’s Drug Store
was black. A lot of sandwich shops were black. My mother [Mary Ella Howze]
and her sister opened a little sandwich shop next door to the Rhumba Theatre for
a time. And they sold hot sausage sandwiches and fish, fried fish sandwiches,
to people going into and coming out of the Rhumba Theatre... And there on
the corner was also the, the Rhumba Theatre with–who, after we watched the
movies, then we would go find George and make him chase us. George was a
person who cleaned up the theater and was also the security...”

b)Bev Smith’s Interview

The radio show host Bev Smith was born in Pittsburgh on March 4, 1943.
She was a radio host for the stations KDKA in Pittsburgh, WGBS in Miami
(now WNMS), and WKIS in Orlando. Smith also served as the thirteen-year
host of the BET conversation show Our Voices. For her services to radio and
television, Smith has won close to 300 honors and awards, including the coveted
Max Robinson Award and the Radio Air Crystal Award. In the conversation,
she spoke about her memories of living in the Hill District and her feelings

31
towards its renovation.[80]

Bev Smith’s Interview Clip Examples:

”...I was on the radio at that time. And I was asking–because I had a talk show
called, ”In the Public Eye”, and I begged people not to sell their homes in the
Hill District [Pittsburgh, Pennsylvania], which is where I live now. This was all
black, black and Jewish, black and German. And this was where the community
was considered the lower Hill. This was the part of the Hill where you had to
throw your hat in, where I’m living right now. You would throw your hat in. It
was the lower Hill, and you didn’t go to the lower Hill. Although you did go,
because they had the best jazz clubs, and stuff like that. And then, yes, the city–
the organizers, the fathers, the developers–started to push and push. And they
started to push blacks out of the Hill District, because they had moved whites to
the suburbs. And the suburbs had high taxes, very little, if any, transportation.
And black people–white people wanted to move back into the city that they fled.
And so, gentrification began, and continues even to this day...”

4.2.2.2 African American Jazz Preservation Society of Pittsburgh

Oral History Project
There are no transcripts available for this oral history resource and all the
documentary resources mentioned in this section. As a result, all of the video
segments are manually translated into texts by myself.

Boswell Rober’s Interview

Boswell Robert used to be a bassist who used to rehearse with Billie Holiday
in Pittsburgh. He talked about how he joined Local 471 and his early musical
experiences in this interview. His memories of the other artists in this musical
society were also brought to mind. He described it as the setting where they
could interact with each other and “got really connected”.[81]

Boswell Robert Interview Clip Examples:

“...Well I will say yes. Because we used to go to sessions down the union.
Everybody could sit in there. There was a room where you could play. We
were all going in and out and played the sessions there because how else could
everybody know what you could do? And that’s how I really got connected...”

Hughes John’s Interview

Hughes John was a teacher as well as a pianist who had performed in Pittsburgh
in the past. He discussed his involvement with Local 471 and his relationships
with other musicians throughout the conversation. He highlighted how wonder-
fully the union’s (Local 471) musicians were taken care of. They had “food,

32
brotherhood” and “business they wanted to conduct”. The union (Local 471)
also provided legal services for the musicians. The musicians received legal as-
sistance from the union (Local 471) as well. The club evolved into a location
where individuals could find support outside of their musical endeavors.[82]

Hughes John Interview Clip Examples:

“... It serves the musicians who actually come into Local 471. For the Food,
for the brotherhood, for the business they wanted to conduct. . . You can come
and get some legal help. You can come in and get some musical help, some
good food, and brotherhood. And also you can meet other musicians. . . It was
great... Yeah they have a kitchen there. If you travel there and you live in
the hotel downtown. You can hook up at night there. . . They have chicken
sandwiches. got some food, socialize you know. They have already got a good
kitchen. . . ... it’s like a club. . . there are plays, practices and talks to find out
what’s going on with the musician who came in”

4.2.2.3 Documentary: The Renewal of Pittsburgh’s Hill District

Prof. Laurence Glasco Interview
Prof. Laurence Glasco was Associate Professor of History at the University of
Pittsburgh. He was born in Xenia, Ohio, in 1940. Since joining the Univer-
sity of Pittsburgh’s history department in 1969, he has concentrated on local
and worldwide African American history. He discussed his views on the ur-
ban renewal project here and the growth of the Hill District throughout the
conversation.[83]

Prof. Laurence Glasco Interview Clip Examples:

“...Blacks got jobs. They won’t get people to reserve a labor shortage. . . And
the same thing happened to women. So you got women moving into the labor
market. Blacks and women really got benefits economically. . . It was very good.
It was very bad. It was something for everything. . . The fall and down of the
Hill is the success of the people. They moved up. They wanted new houses. So
they became increasingly impoverished... ’Look out! This is all new.’ You know
that Americans always have this idea that something new is gonna be better.
So this is a new idea, the new apartment, new Civic Arena, a new image and
the other. So this just fits in with the American idea that new will always be
better. . . Really the rights on the top of urban redevelopment is what made the
Hill District a vibrant neighborhood...”

33
4.2.2.4 Documentary: Civic Arena Engineering and Innovated in
Pittsburgh
The clips in the documentary contains some discussion and interviews among
Rob Hoffman (an architect and an advocate for saving the arena), John Minetti
(a structural engineer) and Fill Hundley (Principal in the architectural firm
DRS) who all have admired the Civic Arena.

Interview and Discussion Clip Examples in “Civic Arena Engineering

and Innovated in Pittsburgh”:
“...The dome design, the cantilever trust that’s supporting the whole thing, I
think probably most engineers, most people who have never heard of it probably
wouldn’t believe that it opened like it did...” (John Minetti)

“...The concept was to have a dome that opened. . . Well how do you get this
dome to open? It’s more than architecture. It’s engineering...” (Fill Hundley)

“...We got people there who were true visionaries here you know and pushed
the envelope on the use of structure and the use of materials and originally the
structure was not even intended for hockey. It was just gonna be the Civic Light
Opera in concerts, sort of like the Hollywood Bowl...” (Rob Hoffman)

“...I went one night to the Civic Light Opera with Mr. Dieter and we went
in there and they opened the dome and it just blew me away. I said thank God
I’m with this firm because this is so great. I’m part of this. I’m part of this
excitement of architecture...” (Fill Hundley)

“...I was there once for a concert also and I don’t know if you had the same
impression but it was silent. I really didn’t hear like all this machinery I mean
it just was moving...” (John Minetti)

“... In many ways this is the right project for Pittsburgh. The steel, the
innovation. . . It’s gonna cost 1520 million dollars to demolish the building. It’s
a lot of money number 1. Number 2 you are losing something you will never get
back. It’s truly a Pittsburgh landmark...” (John Minetti)

4.2.2.5 Documentary: Pittsburgh People and Places Tour

Clip Examples of Prof. Laurence Glasco’s Introduction about the
Freedom Corner
”...Actually it had been called Freedom Corner for some time because it started
in the mid of 1960s when people used to gather right on this spot to start marches
towards downtown to protest of all sorts of things. Particularly they protested
jobs. They protested for housing. They protested against police brutality. They

34
protested against the war in Vietnam. It became a rallying place because of its
location right towards downtown... This was the place that the community, the
hill district community, came together to protest what was the greatest probably
tragedy that ever to befall the Hill and that is what was called Urban Renewal.
That is the redevelopment of the Hill which involved tearing down a huge ex-
panse of the commercial center of the Hill for the direction of the Civic Arena.
The idea was that this was to be a benefit to the city and the community. But
it turned out not to be so... This area really represents a continuing sense of
protest and search for racial, social, economic, gender justice. It’s not just
Blacks protest from this area. But women protested for women’s rights. Gay
protested for their rights. All sorts of groups who feel that the city, the state, the
nation needs to address their needs have used this area as a symbolic gathering
place to move forward...”

4.2.3 Other audio resources

4.2.3.1 BBC Sound Effects
Beginning in 2018, BBC has made 16,000 BBC Sound Effects available for down-
load in WAV format online in accordance with the RemArc Licence (used for
personal, educational or research purposes). All sound effects could be searched
in designated groups and attributes thanks to the meticulous database creation.
I acquired sound effects from this database, such as the sound of building con-
struction and protester’s’ voices.[20]

Figure 20: The BBC Sound Effect data set, reproduced by the author from
”BBC Sound Effects” website[20]

35
4.2.3.2 “La Space”: Album of Stanley William Turrentine
Tenor saxophonist Stanley William Turrentine played jazz in America. Turren-
tine was born into a musical family in Pittsburgh’s Hill District, in the United
States. Mr. Turrentine never lost touch with his origins in Pittsburgh. His
record from the middle of the 1980s, ”La Place,” was named after a street in
the Hill District and the honors listed the names of a number of dozen indi-
viduals, some of them were artists he knew growing up.[84] The audio clips are
taken from Stanley Turrentine’s album “La Place”.[21]

Figure 21: the playlist of “La Place”, reproduced by the author from AllMusic
website[21]

36
4.3 System Building
4.3.1 Pretest demo
To test the virtual assets displacement in the physical place, I designed a pretest
project based on 2D historical images and oral history audio clips. Adobe Aero,
which offers a simple method to create, watch, and share immersive narrative
experiences in mobile augmented reality, was used to create this project. With
Adobe Aero, it’s simple to quickly create an augmented reality scenario with
multimedia content and interactivity. Additionally, designers have the option
to perform rapid iterations with the asset placement and experience design pro-
cesses on the site using the program. Figure 21 depicts the Adobe Aero user
interface.

Figure 22: Adobe Aero interface (Made by the author)

I created a prototype using Adobe Aero that features three scenarios to show
neighborhood civic engagement, urban regeneration initiatives, and local eco-
nomic activity. Materials are gathered from the audio recordings from section
4.1.3 and the historical photos referenced in section 4.1.1. By projecting visual
components on specific sites where the events took place in the past, this pretest
prototype replicates a location related augmented reality experience. Addition-
ally, I tested the audio play that is linked to the picture and is activated when
the user taps the image. Figure 23 displays the screenshots from the pretest
demo.

I went through multiple iterations regarding site selection and material dis-
placement based on this pretest, which really aids in the scenario design. Addi-

37
tionally, it offers the opportunity to swiftly alter the contact process. In section
4.3, these improvements will be covered in more depth. Even though Adobe
Aero provides a perfect tool for fast prototyping, it’s hard to build a location-
based augmented reality project since geometry data can’t be attached to a
specific location according to location tracking sensors without image anchors.
Therefore, for the following prototyping and testing process, I will switch to
Unity engine which will be introduced in detail in sections from 4.2.3 to 4.2.9.

Figure 23: Pretest demo screenshots to display local economic activities, the
urban renewal program and civic activities around the Freedom Corner (Made
by the author), with materials from historical photo collections[17][18]

38
4.3.2 Modeling and digital site building
4.3.2.1 Constructions
Previous constructions as an important part of the heritage scene provides a
basic background for human activity scenarios and audio play. Based on the
analysis in section 2.3.4, six sites are selected as follows: Goode’s Pharmacy,
Local 471’s hall and Musician’s Club, Rhumba Theater and a nearby sandwich
shop, Bethel A.M.E Church, Civic Arena and the Freedom Corner. All photos
of constructions are shown in Figure 24.

Figure 24: All constructions within the project (Made by the author)

Historical maps and photographs serve as the foundation for architectural

modeling. Consider the Rhumba Theater as an illustration. I investigated the
historical map of the Lower Hill District in 1923[22] to set the model scales
as seen in Figure 25, in accordance with its recorded address (53 Fullerton
Street). Additional modeling information is gathered from old photographs
taken from various viewpoints (Figure 26). The model’s textures are chosen
from the Blender texture collection.[85] Historical images are sometimes utilized
as textures for displays and signs (Figure 27). To ensure that the texture will
be properly shown for the subsequent processes, each architecture is exported
from Blender to Unity independently with a texture folder.

39
Figure 25: Model building based on historical map (Made by the author), the
map which is used as a reference is a historical map of Pittsburgh in 1923[22]

Figure 26: Model building based on historical photos (Made by the author), the
historical photos used as references (right) are from historical photo collections,
photo by Charles “Teenie” Harris[17][18]

4.3.2.2 Human Activity

Human activities is another layer to be built in this application. At first I chose
to build 3D models based on pre-designed low-poly human assets from Unity
Asset Store[86] as the method to represent local human activities. The mod-
eling and editing of human activity assets are based on historical photos. It

40
Figure 27: Texturing process with textures from historical photo (Made by the
author)

takes into account the inhabitants’ physical characteristics and attire from the
photographs. Figure 28 displays a comparison between historical images and
model assets. The production process of 3D human assets contains three steps:
model editing, armature attachment and pose editing.

41
Figure 28: Comparison between historical images and model assets (Made by
the author)

However, I found that these low-poly models were not realistic enough to
reflect previous community members and their activities. Therefore, 2D pho-
tographs are also modified and added to the environment using Adobe Photo-
shop. Figure 30 illustrates how the background of the photographs are removed
to allow for a more seamless integration between 2D human figures and 3D
architectural settings.

42
Figure 29: The Armature Attachment process of human assets (Made by the
author)

4.3.2.3 Other Assets in the scenario

In addition to buildings and people, the augmented reality application also uses
certain additional model assets. These models are relocated in the scenarios to
improve the narration, such as the cranes within the scene of the Civic Arena.
The models presented in the section were all obtained from Sketchfab[87] and
are all listed in the appendix.

Figure 30: 2D assets used in the augmented reality scene (Edited by the author),
with materials from historical photo collections[17][18]

43
4.3.3 Devices
The equipment needed for this augmented reality application is a mobile phone,
which includes computing power, positioning sensors, portable screen, a cam-
era. Headphones are also required for audio listening. Since this program was
created in Unity utilizing the AR Foundation SDK for the Android platform,
it is strongly advised that the user select a mobile device running the same op-
erating system in order to install the application. Regarding the headphones,
high-quality binaural headphones might offer a better presentation of all ambi-
ent audios throughout all situations. However, users should be cautious while
using headphones with a significant noise-canceling feature for this project since
they might miss all of the outside ambient voices from the physical world when
they take the location-based augmented reality walk.

4.3.4 Tracking and Registration

This project uses sensor-based tracking as its tracking technique. Sensor-based
tracking is more suited for situations with high levels of spatial and visual com-
plexity, such as most metropolitan environments, as opposed to camera-based
tracking, which depends on the right lighting conditions and easily-recognized
physical features. Three different types of sensors, including a global position-
ing system which provides the user’s current location information and serving
as the system’s starting point, a magnetometer which supplies the user with a
real-time orientation guide and determining the placement of virtual items, and
a gyroscope which gives the virtual camera and sound receiver adjustable real-
world direction, are employed specifically for this project to detect the location,
rotation, and motion of the user in the real environment.[88] Smartphones used
for this project should contain all three kinds of sensors mentioned above.

In terms of registration, location-based initiatives need a more adaptable

and portable engagement procedure than conventional marker-based solutions.
As a result, all operations in this project are completed based on game object
interaction in Unity platform.

4.3.5 Location-based Asset Display

The AR Foundation SDK and the AR+GPS plugin[89] in the Unity engine were
used to create the project’s whole location-based functionality. After importing
the plugin package, an object called “GPS Stage Object” can be appended in
the scene and a script named “Place at Location” is automatically attached.
Then, as illustrated in Figure 32, it is simple to obtain a set of latitude and
longitude data by choosing a specific location site on the Google Map. When
materials are added to the ”GPS Stage Object” as child objects, all assets,
including the visual and audio ones, will be put on the specific address. The
”Altitude Mode” parameter should be changed to ”Ground Relative” to ensure
that the assets are positioned at the proper altitude. To conceal the entire ini-
tial distance tracking interface for the visual assets, ”Debug Mode” should be

44
disabled after the testing process.

Figure 31: The inspector setting screenshots for location setting in Unity (Made
by the author)

45
Figure 32: Getting latitude and longitude data of geography location from
Google Map (Made by the author)

4.3.6 Audio Elements Settings

4.3.6.1 Audio Clips Importing
Maintaining a balance between file size and quality is crucial to provide a fluid
experience on mobile devices. As a result, it’s crucial to regulate the import-
ing procedure for audio assets, particularly the parameter of format and load-
ing technique. The audio file forms that Unity can accept are.aif,.wav,.mp3,
and.ogg, according to the Unity audio importing guide.[90] Since the .mp3
file employs “lossy compression” to encode audio data and has a considerably
smaller file size than any other kind, I decided to use it for all audio parts in
my project. Also the .mp3 files will load faster when played because of their
compression format.

The Unity engine supports three distinct audio loading modes, including
”Compressed In Memory,” ”Streaming,” and ”Decompress On Load,” as well
as three different compression types, including ”PCM,” ”Vorbis,” and ”AD-
PCM.” According to Korzuszek’s blog post regarding the audio import settings
in Unity, it is preferable to use ”Compressed In Memory” as the load format for
seldom played and brief audio clips, ”ADPCM” for oral history documents and
”Vorbis” for environment audio clips as compression format.[91] This is due to
the fact that the duration of the environment audio clips in this project ranges
from 3 to 5 minutes while the duration of the oral history audio clips is between
60 and 90 seconds. In comparison to raw ”PCM,” ”ADPCM” is around 3.5

46
 Figure 33: The audio clips import settings in Unity (Made by the author)

times smaller, and the decompression process uses less CPU power than ”Vor-
bis”. Nevertheless, lengthier audio snippets can be too lengthy to apply the
”ADPCM” approach.

4.3.6.2 Audio Spatializer Settings

Unity offers spatializer effects that can include the two-eared ”head-related
transfer function (HRTF)” to support the audience better perceive the orien-
tation of the audio through the noise removal of the head and the small delays
between the ears.[92] This enhances the rationality of sound extension. Customs
can click the ”Audio” section inside the ”Project Setting” panel as illustrated
in Figure 34 to add this extra customized spatialization effect. Then select ”MS
HRTF Spatializer” from the drop-down option to install the ”Spatializer Plu-
gin”. After the plugin has been installed successfully, customs may turn on this
feature by checking the ”Spatialize” box on the ”AudioSource” panel.

4.3.6.3 Audio Play Settings

In order to play sound in a physical domain, game objects in the Unity engine
are associated with a ”AudioSource” attribute. While an audio file is added to
the ”AudioClip” category in the “AudioSource” panel, it automatically plays

47
Figure 34: The spatializer plugin install process (Made by the author)

48
with “Play On Awake” and “Loop” checking on. So the ”Loop” selection needs
to be turned off initially since it will triggered only after the user tap on the
start button in the scene. Also the “Spatial Blend” needs to be set as 1 to
turn on the 3D sound effect. Besides, the “Min/Max Distance” section can be
changed to adjust the audio’s audible hearing distance[93].

Final setting in for “AudioSource” of this project is shown in Figure 35.

Figure 35: Parameter settings of “AudioSource” in Unity (Made by the author)

49
4.3.7 Interaction Settings
There are two interaction design targets in this project. One is to let the user
tap one specific object in the AR scene. The other is have a clickable icon on
the interface for the mini-map. Therefore the interactive settings in Unity will
also be divided into two sections in the following discussion: interaction in the
augmented reality scenes and interaction on the user interface.

Interaction in the Augmented Reality Scenes

The essential design problem of this section is to figure out a way to detect the
tabbing screen activities from the user. If the clicking point is on the game
object, then the relatives events will be triggered. I set a ”RaycastHit” variable
as ”Hit” and the position value called ”Camera.main.ScreenPointToRay (In-
put.GetTouch[0].position)” as ”ray”. Based on Unity documentations about the
function called ”Physics.Raycast”[94], I chose to use ”Physics.Raycast (ray,out
Hit))” as a test value to verify if there is a hit from screen. Once a hit is re-
ceived, the function will then check the ”Hit.transform.name” to see if it’s equal
to ”button”, which is the object category name I set for the interactive button
in all the scenes. By doing this, once the user click on the button on the mobile
screen, the interaction will be automatically triggered.

There are two following events after the button clicking has been checked as true.
The first one is toggle the visibility of human activity images and the interactive
button. After the clicking, the images of human activities will be shown in the
scene and the button will be hidden. I made this happened by changing the value
of the interior function in Unity called ”GetComponent¡Renderer¿().enabled”.[95]
The second one is the audio clips of environment sounds which are attached to
this scene will be played. The method of playing specific audio in the back-
ground is using ”Getcomponent¡audiosource¿().clip”[96] to set the audio to be
played and using ”Getcomponent¡audiosource¿().play()”[97] to play the specific
clip.

Interaction on the User Interface

As for the interface interaction part, I chose to set the mini-map as a ”But-
ton” object which has been pre-designed in Unity.[98] I set all the click events
based on the properties of the ”Button” control. Based on the ”Button.onClick”
function, the click event can be detected and responded through tracking the
”TaskOnClick” status. The final interaction about the mini-map is the user can
click the button to open and close the mini-map. The yellow spots on the map
illustrate the locations of all sites in the project.

50
Figure 36: Screenshots of the interface with the mini-map included (Made by
the author)

4.4 Experience Design

The experience design process is essential to this study’s attempt to address the
research topic of how location-based augmented reality may be developed as an
on-site preservation strategy to influence users’ awareness of intangible assets.
In order to improve the authenticity and deliver-ability of the story, the main
focus of this section will be on how to physically depict digital architecture,
human activities, and auditory features. The following factors were taken into
account when I developed this technique: content presentation, scenario design,
and user flow design.

4.4.1 Content Presentation

As stated in section 4.1, the project’s materials are split into three groups: con-
structions, human activities, and audios (environment sounds and oral history).

51
When I tried to map every object in the physical world, ”location-based” was
the crucial notion. Digital representations of buildings are positioned on their
historical addresses in accordance with the historical evidence from old pho-
tographs and other public sources. In order to better illustrate the social and
cultural activities in the community, which can be considered as an important
part of the intangible cultural heritage, 2D and 3D assets of human activities
are placed around buildings or structures based on the explanations in related
oral history materials. Thirdly, to further enhance the immersive nature of the
project, audio excerpts featuring interviews and environmental voices are also
played. More information about historical occurrences and everyday life in com-
munities can be found in oral history recordings. The following graphs depict
the spatial organization of all the contents, and the appendix has a complete
list of all the digital materials.

Physical environment, digital structures, displays of human activity, and

ambient audios are intimately integrated to strengthen the story of scenes.
Consider the Rhumba Theater scenario as an illustration. The construction
of Rhumba Theater and next-door sandwich shop are placed on their previ-
ous location on the site. According to some public historical archives like Sala
Udin’s oral history, ”...And there on the corner was also the, the Rhumba The-
atre with–who, after we watched the movies, then we would go find George and
make him chase us. . . My mother [Mary Ella Howze] and her sister opened a
little sandwich shop next door to the Rhumba Theatre for a time. And they sold
hot sausage sandwiches and fish, fried fish sandwiches, to people going into and
coming out of the Rhumba Theatre...”, photos of staff and local artists who used
to work or perform here are included into the setting based on the description.
Additionally, a busy business scene ambient sound from BBC Sound Effects is
used to provide a more vivid picture of earlier cultural and economic facets of
community life. To provide users a deeper understanding of what occurred here
in the past, Sala Udin’s oral history interview are also played in the background.

4.4.2 Design Scenarios

From the perspective of narrative, the six sites can be divided into three groups
based on their historical following the sequence of historical periods. These
three scenarios address several facets of intangible heritage in the Lower Hill
District’s local community, including the thriving economic and cultural lives
in the neighborhood, the impacts of urban renewal projects, and civic rights
activities around Freedom Corner.

Here are some further descriptions of each scenario.

52
list.png

Figure 37: Asset Distribution (Made by the author)

Figure 38: On-site augmented reality scene of Rhumba Theater (Made by the
author)

53
Scenario 1 (site 1-3):
This scenario depicts the neighborhood’s appearance in the 1940s. Both the
artistic and economic life of Pittsburgh’s Lower Hill District was centered around
Fullerton and Wylie. William Goode, a pharmacist, was standing outside of
Goode Pharmacy. The neighborhood was well known for local businesses like
Goode’s Drug Store. Selling hot sausage sandwiches and fish sandwiches to
those entering and exiting the Rhumba Theater, the sandwich shop beside the
door was booming. The American Federation of Musicians’ Local 471 office was
housed in a bar at 1213 Wylie Avenue. Jazz musicians practiced and performed
here.

Scenario 2 (site 4-5):

The second scenario displays the changes that happened in the neighborhood
during the Urban Renewal project from the 1950s to 1960s. Bethel A.M.E
Church was removed from the district for the construction of Civic Arena. On
the same location, the Civic Arena was built for public activities including mu-
sical performances and sport competitions. Engineers, works and constructing
tools are shown in this scenario.

Scenario 3 (site 6):

A billboard was placed at the intersection of Crawford and Centre Avenue,
which said “Attention City Hall and U.R.A.: No Redevelopment Beyond this
Point!/We Demand: Low Income Housing for the Lower Hill”. People gathered
here to express their feelings about stopping further redevelopment in the dis-
trict and get lower price housing back. The sound of the protest can be heard
in the background.

4.4.3 User Flow Design

As shown in Figure 39, the user flow of the project starts with checking the
mini-map and moving towards the site. Once users enters the site within the
pre-designed distance, they can view the model of historical constructions and
hear the oral history narratives in the background. Next, when the user clicks
the start button in the scene, the photos of human activities will be displayed.
At the same time, enlivenment audios like jazz music or the sounds of protests
can be sensed by users. After they finish the experience of the site, users can
double check the mini-map and head for the next site.

54
Figure 39: The framework of user flow in the augmented reality walk (Made by
the author)

55
5 Evaluation
To better evaluate how the location-based augmented reality project affect peo-
ple’s feeling and understanding about local heritage, I designed a user test with
three parts: site walk, individual interview, take-home survey. I had 10 partic-
ipants in total who are students with design or urban planning backgrounds or
augmented reality developing experience. The test and evaluation focus on their
opinion about the design of hybrid walk on the site, multimedia display as well
as how this experience changed their perception of local heritage. After having
a hybrid site walk with their own mobile phones and headphones on the slide
walks around the Freedom Corner in the Lower Hill District, I will have a in-
terview with each participants individually. Then for the last step, a take-home
survey in digital version will be given to all the participants to collecting their
views about multimedia design and interaction design more specifically. The
reflection and evaluation of this test provide valuable references for the future
iterations about the hybrid walk experience about intangible heritage based on
augmented reality platform from the design perspective.

5.1 User Test

The purposes of all the three activities, site walk, individual interview, take-
home survey, within this user test are the followings:
• observe how participants behave/think/feel during the on-site augmented
reality walk
• observe how they interact with the digital heritages and how they get
affected
• analyze how the visual and audio elements in the augmented reality ap-
plication affect people’s feeling and understanding about local intangible
heritage

5.1.1 Overall Process

Participants were invited to the site in the Lower Hill District to experience
the location-based augmented reality application with their own mobile devices
and headphones. Potential participants include people with extended reality de-
velopment experiences or have a related design or urban planning background
to provide professional perceptions for future iterations and research. The ex-
pected number of participants was 10 (divided into 2 sessions, about 5 for each
session). The application was installed on the devices before the site walk starts.
A test demo was run on their devices to make sure they won’t have any technical
concerns during the walk. If the participants have any privacy concerns, they
can choose to use my own mobile phone to get involved.

56
Participants went through the experience one by one for around 30 minutes. As
participants took the walk, they were asked to think aloud which means they
were required to talk about what they were thinking or feeling during the walk.
The talking process was documented under their permission.

After the walk, I had a 10-minute interview with each participant. The inter-
view was also documented. Afterwards I gave each participant a take-home
survey (digital version).

Figure 40: A participant was viewing the virtual billboard and people in the
protests on the historical site(Photoed by the author)

57
5.1.2 Interview
The interview was held on the sidewalks around the Freedom Corner. After
having a view of the intangible heritage of the district digitally, participants
accepted the interview within the physical space individually. The reason why
I want to avoid making the interview session as a focus group discussion is that
individual’s personal understanding could be affected by other’s thoughts or
feelings. It could be harder to collect their initial thoughts and feedback if they
have exchange their minds. Therefore, the interviews were held and documented
separately immediately after the side walk. The interview transcripts can be
found in the appendix.

The topic are covered in the interview:

• Reflection of the experience
• Immersive level of the overall augmented reality walk

• Feeling and understanding about the district after the walk

• Other comments or suggestions

5.1.3 Survey
The take-home survey was provided to participants after they finish all the
on-site activities. Based on this survey, participants could provide more com-
prehensive viewpoints after they had more time to review the whole experience.
Besides, this survey focused on some more specific questions like how well the
multimedia elements affected their understanding process.

Quantities questions:

• How well do you think the visual elements help with your understanding
about the heritage?
• How well do you think the audio elements (environment audios and oral
histories) help with your understanding about the heritage?
• How well do you understand the historical time period shown in the ap-
plication?
Qualitative questions:
• How do you feel about the interaction process of this augmented reality
application?

• What do you think about the correlation among heritage sites?

58
Figure 41: A participant was taking the interview after the augmented reality
walk (Photo by the author)

59
5.2 Data Processing
The data processing progress was divided into two parts: organizing data and
understanding data. For the organization part, I started by combining records
from different parts of the test of the same participant together and replacing
all the identical information like names with the assigned tester number. After
that qualitative data and quantitative data are separated for further data fa-
miliarization progress.

For the second part, I generated raw data for each participant and data visual-
izations as pie charts for each question with the Google Form response collection
tool. All further data evaluation will be built on those results. As for the quan-
titative data, I decided to go through all the documented interview scripts first
and got familiar with the patterns and commons for each question. Then I
tried to map all the answers with research questions mentioned in the previous
sections.

5.3 Data Evaluation

In this section I will go through different kind of data I gathered from 10 par-
ticipants about their thoughts and comments of the whole augmented reality
walk in multiple aspects, including immersive experience, visual and auditory
experience, and supports for understanding of the local heritage.

5.3.1 Multimedia Assets Display

”...For me I think the visual parts works better. I mean it’s hard for you to see
the digital views like this. You can go to the museum if you want to know more
about this piece of history, but it’s very hard to gain an immersive experience
like this. The scale of the architecture and people are real. The textures are
real. At least it can make you more impressive. As for the audio, I think they
are related to the visual elements right now. But I think if you can make it
more like a dialog, you know. Like in a game, I can click on a character, and
make him talk to me. I think it will definitely make the whole experience more
interesting.” (Participant 1)

”...The audios worked well. I learned a lot from the narrator in the background.
As for the digital models, they are not bad. But I think they looked not that
real. They still looked like models...I think it’s their textures. You know when
we normally render a model, the texture assets really matter. So I just wonder
if you can modify this issue by changing the textures or rendering materials you
choose...” (Participant 5)

”...As I said, I liked the visual elements since they are nicely built and positioned
on the location. However, I have to say I learnt more from the audio. I mean
the speaker in the background. Yes the oral history. The oral history provided

60
more details. Otherwise I couldn’t fully understanding what happened here...”
(Participant 8)

Generally participants thought all visual and auditory elements are relevant to
the heritage and local history. The real scale and historical based position make
the experience more realistic. It makes users easier to image how community
used to look like previously. Since visual elements are more ”intuitive” than
aural storytelling, some individuals prefer them. At comparison to traditional
media like video records or displays in museums, the portrayal of physical and
intangible heritages through augmented reality in real space also provided them
with a more immersive experience. When participants approached the digital
buildings and saw the physical space at the same time, they will immediately
built a connection between the past and present of the community. It’s hard to
view the heritages which were removed from the site through other approaches.

For another thing, some participants, like participant 5, learned more from the
audio components like the oral history. They believed that the audio tales of-
fered a lot more information to aid in their understanding. As Participant 8
mentioned in the interview, it’s hard for him to fully understand what happened
if he didn’t hear the oral history in the background. Participant 10 and others
who shared his perspective said that the project’s integration of the visual and
audio elements greatly enhanced their ability to see and comprehend the his-
torical era and daily lives of the community.

On the other hand, some participants mentioned that the digital scenes on
several separated spots make the experience intermittent. As participant 10
said, she believed it would be easier to make the experience for her feel more
real and immersive if she could see a street rather than a single building. These
viewpoints suggest that the quantity and distribution of digital assets in the
physical world will have an impact on people’s perceptions of the degree of
immersion of the hybrid experience. Additionally, it is anticipated that the
portrayal of designs will be improved because, as several participants noted,
digital architectures still resemble ”models” more than actual architectures. The
need to increase the realism of scenes in upcoming research and projects arises
from the fact that this is a reflection of the historical context.

5.3.2 Understanding about the district and local heritage

”...Yes I think they helped. Previously I didn’t know much about the district.
But when I saw the images of people who used to live here, I started to realized
here might be a living neighborhood before. I tried to figure out the reason why
they disappeared in the following process. It’s pretty nice, the exploration based
on all the oral history and visual cues...” (Participant 6)

”I don’t know much about the relationship among heritage sites. If some vi-
sual cues can be offered or there is a correlation among the digital contents of

61
different sites. In the AR environment, I feel like I just encounter them when
walking around and someone introducing the heritage by showing something. I
won’t pay much attention to understanding the relationship between them. If
there are some interactions of feedback, users might have more motivation to
try to understand that.” (Participant 9)

This section of the evaluation discusses how augmented reality aids in the
comprehension of regional heritage and history. The third and fifth survey
questions as well as the interview subject concerning personal understanding
are all part of the information presented in this section. People felt that the
interview and open-ended survey questions contributed to their awareness of the
region’s history and cultural heritage. Even while some participants complained
that it was difficult for them to recall every detail once the tour was over, they
still felt that the augmented reality experience at such important historical
sites piqued their curiosity about the community’s intangible legacy. I assess
their comprehension of the connections between all the heritage sites from their
descriptions. For this section, half of the participants believed they could sense
the association from the standpoint of community development. The others
believed it would be better if there were interactive elements to boost their
interest for learning more about it. They felt it was somewhat difficult for them
to understand the relationships across various sites.

5.4 Interaction
”...I think the interaction process is pretty direct and intuitive. I have really
enjoyed it. What was particularly interesting to me is that there was a thing
there is several workers involved in the sandwich shop and I think this thing is
really good because is there a lifelike and there’s this sort of tenderness in it and
it feels very relatable. I really like this project...” (Participant 7)
”... I think it’s necessary to try to add more interactions. Sometimes I can’t
help clicking different elements. If I can get some reactions it will definitely be
more attractive to me...” (Participant 6)
”...The AR application helps me understand the physical heritage better. It in-
tegrates different forms of media, providing a sense of immersion. But I think
there can be more interaction mechanics to strengthen the user’s participation,
like how the users trigger the oral histories. It would be more impressive if the
digital content can respond to users’ behavior...” (Participant 10)

As for the interaction design, most participants thought that it makes the
whole experience more engaging and interactive. The map leads users to move
around the site. But some participants like participant 3 thought that the size
of the map is too tiny so it’s kind of hard for them to read the map. Also many
participants mentioned that they were willing to see more different kinds of
interactions among digital assets. For instance, participant 10 expected to see
clickable architectures to make the scenes look more real and attractive. Other
comments also mentioned some methods like adding certain real-time features or

62
integrate the audio components with the human figure more closely to improve
the project’s interaction and create a hybrid environment that is more engaging
and immersive.

63
6 Discussion
6.1 Intangible Heritage Representation via Multimedia
One of the major goals of this research is to depict local intangible heritage,
which includes local cultural, social, and economic activities, utilizing multime-
dia, including visual, audio, and interactive aspects. In the past, academics in
this sector focused more on the preservation of tangible legacy or the absence
of vibration in the exhibition of intangible heritage.[45][46] Heritage, as I have
stated, is more than simply an item. To portray local intangible heritage, the
historical setting, the physical subject, and the regional customs and practices
should all be merged as an integrated whole. Together, they may give the
audience a better and more complete understanding of the heritage’s origins,
historical development, and present-day significance for the neighborhood.

I proposed a ”location-based” augmented reality experience as a strategy for

representing regional heritage in order to do this. The term ”location-based”
denotes two levels of information. For starters, the hybrid experience may ac-
cess the actual location information because the augmented reality tour is taking
place on a historical landmark and area. On the other hand, depending on loca-
tion data, all digital assets, such as models, photos, and audio files, are mapped
to the actual physical place. For instance, the model of Goodie’s Drug Store,
photographs of Mr. Goodie operating the store in the 1940s, and oral histories
of the drug store and other local businesses from local residents are all combined
and merged as a complete intangible heritage scene and are relocated on their
historical grounds.

According to the user test, the integration and connection of all the items sup-
ported the participants’ comprehension of the regional heritage. When depicting
historical settings and human actions, the visual aspects are intuitive. The au-
ral components, such as oral history and background music, provide further
information about the historical events. Users can form a vivid and thorough
impression in their brains by investigating the link between visual and audio
elements.

6.2 Design Implications

The methodology for designing and creating a location-based augmented reality
experience for a community’s intangible heritage is another contribution of this
study. Data gathering, digital construction, experience design, and application
development are the four processes that make up the entire design flow. The
process of gathering data include collecting various types of historical content
from a variety of sources, such as pictures, audio snippets from oral histories,
documentaries, etc. Creating relationships with local authority groups like the
local library, the government, or related public archive organizers is an efficient
way to obtain relevant material for the specific urban space. Another effective

64
approach to get oral history recordings for the project is to gain access to relative
researchers or local community members. Regarding the digital construction,
I made the decision to create virtual replicas of physical cultural assets like
buildings and structures and to reflect human activity on the site by editing old
photographs. All the assets are mapped out as part of the experience design,
and the user interface is created so that users can access them. For instance,
the user in this project will first observe the architectural structure and hear
the introduction to the oral history. They can then press the start button to
view the human photos and hear the ambient sound. Participants had the op-
portunity to gradually increase their comprehension of the site thanks to this
interaction design.

In the last segment, which deals with application development, the Unity engine
plays a crucial role since it offers an environment for creating augmented reality
applications called the AR Foundation as well as plugins that already have
GPS-based location-based services built in. In order to make the entire on-site
walk more immersive and engaging, I demonstrated how to merge digital assets
with actual space as well as how to create the interface and acoustic settings.
Generally speaking, it offers a way for designers with less experience to create
and construct the application with fewer technological challenges.

65
7 Future
7.1 Current Limitations
7.1.1 Technological Limitations
• Considering the limitation of the GPS recognition accuracy(5m-10m), the
assets are supposed to be placed with a distance to avoid the overlap.
Therefore, currently it’s hard to design a continuous scene in a larger
scope.

• The current application is not able to collect user data during the walk. So
the user-test methods are limited to subjective approaches like interview
or surveys.

7.1.2 Design Limitations

• Due to the time and research scope limitation, the interaction design is
simplified as the toggle of visibility of human activity assets in the scene.

• The guiding interface is currently designed as a mini-map to suggest the

user follow the designed walking route. But it’s also possible to design
a location-based notification system to make the whole experience more
immersive and flexible.

• User-test collected opinions from people with relative experience or knowl-

edge now. It’s also interesting to explore the feedback from local commu-
nity members or other stakeholders.

7.2 Future works

7.2.1 Technological Improvements
• In the future, I’d like to explore the methods to refine the accuracy of GPS
recognition to enlarge the scope for heritage scene display in the urban
space.

• Also, data tracking and collection can be combined with user activities
like moving route, dwell time as well as interactive activities. Those data
can be used to a more objective evaluation and help with further design
interaction.

7.2.2 Design Improvement

• It’s possible to exploring the interaction with different kinds of assets like
the architecture models and the human activity images to make the whole
experience more interesting and immersive.

66
 • Also, the guidance system can be refined based on the location-based
technology like triggering an audio when the user approach a specific spot
to guide further direction.
• Besides, under the evaluation section, I’d love to hear more opinions from
local communities and relative authority organizations which can bring
suggestions for project development from different aspects. It’s also possi-
ble to introduce the voice of local communities in the heritage protection
field by letting them investing in histories by using some crowd-sourcing
data about local history.

8 Conclusion
Generally the project provides a more immersive and intuitive method for people
to interact with local intangible heritage and build a better understanding about
it. The multimedia platform worked well to provide a more comprehensive
recognition of the heritage and history. People thought the realistic photos and
the oral history are more convincing than a simple background narrator. It’s
hard for people to understand and remember all the materials during the walk.
But it’s considered to a good way to trigger their interests about local heritage
and community. Also it provides a workflow for future researchers and designer
to develop a location-based augmented reality application with less technological
and design difficulties and challenges.

67
References
[1] Răzvan Gabriel Boboc, Florin Gı̂rbacia, Mihai Duguleană, and Aleš Tavčar.
A handheld Augmented Reality to revive a demolished Reformed Church
from Braşov. In Proceedings of the Virtual Reality International Conference
- Laval Virtual 2017, VRIC ’17, pages 1–4, New York, NY, USA, 2017.
Association for Computing Machinery.
[2] Fabrizio Natta and Michele Ambrosio. AR for Demolished Heritage: the
First Italian Parliament in Turin.
[3] Derek G. Handley. “The Line Drawn”: Freedom Corner and
Rhetorics of Place in Pittsburgh, 1960s-2000s. Rhetoric Re-
view, 38(2):173–189, April 2019. Publisher: Routledge eprint:
https://doi.org/10.1080/07350198.2019.1582239.
[4] Jiyoung Kang. AR Teleport: Digital Reconstruction of Historical and
Cultural-Heritage Sites for Mobile Phones Via Movement-Based Interac-
tions. Wireless Personal Communications, 70(4):1443–1462, June 2013.
[5] Matthew L. Vincent, Mariano Flores Gutierrez, Chance Coughenour, Vic-
tor Manuel, Lopez-Menchero Bendicho, Fabio Remondino, and Dieter
Fritsch. Crowd-sourcing the 3D digital reconstructions of lost cultural
heritage. In 2015 Digital Heritage, volume 1, pages 171–172, September
2015.
[6] Feng Shui Art.
[7] Ozeni Athanasiadou Droumpouki, Anna Maria. Memories of the Occupa-
tion in Greece, May 2016.
[8] Derek Ham. I AM A Man | VR Civil Rights App, January 2018.
[9] Fabrizio Natta and Michele Ambrosio. AR for Demolished Heritage: the
First Italian Parliament in Turin. page 121. September 2022.
[10] Hill District (Pittsburgh), November 2022. Page Version ID: 1123403185.
[11] Shake dancer, possibly “Gypsy Rose Lee” performing with band, including
Tommy Turrentine on trumpet, at Bambola Social Club | CMOA Collec-
tion.
[12] Pharmacist William Goode, standing outside of Goode Pharmacy with
advertisement for Nucoferrin in window, Wylie and Fullerton Avenues, Hill
District | CMOA Collection.
[13] Interior of Musician’s Club, Local 471, with striped awning over bar, tele-
vision in corner, and round bar stools | CMOA Collection.
[14] Exterior of Rhumba Theatre seen from across street, Fullerton Street, Hill
District | CMOA Collection.

68
[15] Naomi Chris Hedlin Harris, Rich Lord. Oldest Black church in Pittsburgh
calls out racism, seeks reparations, April 2021.
[16] Pittsburgh Civic Arena under construction | Historic Pittsburgh.
[17] Pittsburgh City Photographer Collection | Digital Pitt.

[18] CMOA Collection.

[19] Details, Sala Udin describes the Hill District community of Pittsburgh,
Pennsylvania.
[20] BBC Sound Effects.

[21] Stanley Turrentine Albums and Discography.

[22] Archives & Special Collections | University of Pittsburgh Library System.
[23] Tearing Down History.

[24] Dennis Rodwell. Conservation and Sustainability in Historic Cities. John

Wiley & Sons, April 2008.
[25] Shalini Panjabi and Tim Winter. Understanding the tensions in place:
Conflict and conservation in Kashmir. Historic Environment, 22(1):19–25.
Publisher: Australia ICOMOS (International Council on Monuments and
Sites).
[26] UNESCO World Heritage Centre. Convention Concerning the Protection
of the World Cultural and Natural Heritage.
[27] Loes Veldpaus, Ana R Pereira Roders, and Bernard J F Colenbrander.
Urban Heritage: Putting the Past into the Future. The Historic Envi-
ronment: Policy & Practice, 4(1):3–18, April 2013. Publisher: Routledge
eprint: https://doi.org/10.1179/1756750513Z.00000000022.
[28] Rebecca Madgin. Reconceptualising the historic urban environment: con-
servation and regeneration in Castlefield, Manchester, 1960–2009. Planning
Perspectives, 25(1):29–48, January 2010. Publisher: Routledge eprint:
https://doi.org/10.1080/02665430903421726.

[29] Rohit Jigyasu. The Intangible Dimension of Urban

Heritage. In Reconnecting the City, pages 129–159.
John Wiley & Sons, Ltd, 2014. Section: 5 eprint:
https://onlinelibrary.wiley.com/doi/pdf/10.1002/9781118383940.ch5.

[30] Liron Efrat. Historical APPistemology: The Mapping of the Expanded

Field of Cultural Heritage AR Apps as a Creative Tool. International
Journal for Digital Art History, (6):3.02–3.15, October 2021. Number: 6.

69
[31] Guzman Molina, P.C., Pereira Roders, A.R., Colenbrander, B.J.F., and
Architectural History and Theory. Bridging the gap between urban devel-
opment and cultural heritage protection. In conference; IAIA14, 2014.
[32] Densho: Japanese American Incarceration and Japanese Internment.
[33] Augmented reality, November 2022. Page Version ID: 1122588432.
[34] https://plus.google.com/+UNESCO. Cultural heritage, November 2020.
[35] A Bibliometric Analysis of Research on Intangible Cultural Heritage Using
CiteSpace - Xinwei Su, Xi Li, Yanxin Kang, 2019.
[36] Michael Dylan Foster. UNESCO on the Ground: Local Perspectives on
Intangible Cultural Heritage. Encounters: Explorations in Folklore and
Ethnomusicology Ser. Indiana University Press, Bloomington, 2015.
[37] Pinar Yelmi. Protecting contemporary cultural soundscapes as intangi-
ble cultural heritage: sounds of Istanbul. International Journal of Her-
itage Studies, 22(4):302–311, April 2016. Publisher: Routledge eprint:
https://doi.org/10.1080/13527258.2016.1138237.
[38] Vincenzo Lombardo, Antonio Pizzo, and Rossana Damiano. Safeguarding
and Accessing Drama as Intangible Cultural Heritage. Journal on Com-
puting and Cultural Heritage, 9(1):5:1–5:26, 2016.
[39] William C. C. Wright and Florian V. Eppink. Drivers of heritage value: A
meta-analysis of monetary valuation studies of cultural heritage. Ecological
Economics, 130:277–284, October 2016.
[40] Sun-Kee Hong, In-Ju Song, and Jianguo Wu. Fengshui theory in urban
landscape planning. Urban Ecosystems, 10(3):221–237, September 2007.
[41] SANG HAE LEE. Feng-Shui: Its Context and Meaning (china). Ph.D.
ISBN: 9798206250084.
[42] Guo-Ming Chen. The Impact of Feng Shui on Chinese Communication.
Communication Studies Faculty Publications, January 2007.
[43] Intangible Cultural Heritage | The National Heritage Council of Namibia.
[44] Gabriele Ivo Moscaritolo. Reconstruction as a Long-Term Process. Memory,
Experiences and Cultural Heritage in the Irpinia Post-Earthquake (Novem-
ber 23, 1980). Geosciences, 10(8):316, August 2020. Number: 8 Publisher:
Multidisciplinary Digital Publishing Institute.
[45] E. K. Stathopoulou and F. Remondino. Semantic Photogrammetry - Boost-
ing Image-Based 3d Reconstruction with Semantic Labeling. ISPRS -
International Archives of the Photogrammetry, Remote Sensing and Spa-
tial Information Sciences, 42W9:685–690, January 2019. ADS Bibcode:
2019ISPAr42W9..685S.

70
[46] Gamze Fahriye Pehlivan. Remains from a Demolished Cultural Heritage:
A Historical House in Karaagac. Art-Sanat Dergisi, (15):249–285, February
2021. Number: 15.
[47] Francesca Cominelli and Xavier Greffe. Intangible cultural heritage: Safe-
guarding for creativity. City, Culture and Society, 3(4):245–250, December
2012.
[48] Aisyah Abu Bakar, Mariana Mohamed Osman, Syahriah Bachok, and Man-
sor Ibrahim. Analysis on Community Involvement Level in Intangible Cul-
tural Heritage: Malacca Cultural Community. Procedia - Social and Be-
havioral Sciences, 153:286–297, October 2014.

[49] IJIH - International Journal of Intangible Heritage.

[50] Marcello Atzeni, Giacomo Del Chiappa, and Jessica Mei Pung. Enhanc-
ing visit intention in heritage tourism: The role of object-based and exis-
tential authenticity in non-immersive virtual reality heritage experiences.
International Journal of Tourism Research, 24(2):240–255, 2022. eprint:
https://onlinelibrary.wiley.com/doi/pdf/10.1002/jtr.2497.
[51] Florent Poux, Quentin Valembois, Christian Mattes, Leif Kobbelt, and
Roland Billen. Initial User-Centered Design of a Virtual Reality Heritage
System: Applications for Digital Tourism. Remote Sensing, 12(16):2583,
January 2020. Number: 16 Publisher: Multidisciplinary Digital Publishing
Institute.
[52] Mariapina Trunfio, Maria Della Lucia, Salvatore Campana, and Adele
Magnelli. Innovating the cultural heritage museum service model
through virtual reality and augmented reality: the effects on the
overall visitor experience and satisfaction. Journal of Heritage
Tourism, 17(1):1–19, January 2022. Publisher: Routledge eprint:
https://doi.org/10.1080/1743873X.2020.1850742.
[53] Loris Sauter, Luca Rossetto, and Heiko Schuldt. Exploring Cultural Her-
itage in Augmented Reality with GoFind! In 2018 IEEE International
Conference on Artificial Intelligence and Virtual Reality (AIVR), pages
187–188, December 2018.

[54] Ajaya Kumar Dash, Santosh Kumar Behera, Debi Prosad Dogra, and
Partha Pratim Roy. Designing of marker-based augmented reality learn-
ing environment for kids using convolutional neural network architecture.
Displays, 55:46–54, December 2018.

[55] Hill District - Pittsburgh Music History.

[56] Bethel A.M.E. Church Historical Marker.
[57] Martin Delany - Pittsburgh Music History.

71
[58] Christopher Lemery. Guides: Pittsburgh Census Information @ Pitt: 1940
Census.
[59] Monticello-Delaney Rifles | The Black Fives Foundation, November 2008.
[60] August Wilson, November 2022. Page Version ID: 1124055345.
[61] History of Pgh Musicians L. 471.
[62] Bambola Social Club - Pittsburgh Music History.
[63] Lilian McKibbin Steiner. ”Sugar Top” and the ”Cobblestone Jungle”: Ur-
ban Redevelopment in Pittsburgh’s Hill District 1955-1959. Thesis, 2010.
Accepted: 2010-08-11T19:23:32Z.
[64] The Story Of The Pittsburgh Neighborhood That Inspired ”Fences” | Na-
tional Trust for Historic Preservation.
[65] Civic Arena - Pittsburgh Music History.
[66] Freedom Corner | PGH Events.
[67] The Pittsburgh Courier from Pittsburgh, Pennsylvania on August 4, 1951
· Page 13.
[68] Publications | PittPharmacy Publications | Page 13.
[69] Musician’s Club of Local 471 - Pittsburgh Music History.
[70] Tony Norman. Tony Norman: The road to reparations at Bethel African
Methodist Episcopal Church, October 2022.
[71] From ”citadel of hope” to parking lot: Pittsburgh’s oldest Black church
demands reparations.
[72] Mellon Arena Arena: History.
[73] Civic Arena (Pittsburgh), November 2022. Page Version ID: 1119501981.
[74] The HistoryMakers Digital Archive (March 17, 2022).
[75] Guide to the African American Jazz Preservation Society of Pittsburgh
Oral History Project Records and Recordings, 1995-1999 AIS.1998.04 | Dig-
ital Pitt.
[76] Pittsburgh HillDistrict. The Renewal of Pittsburgh’s Hill District, June
2014.
[77] pitcitywatcher. Civic Arena Engineering & Innovated in Pittsburgh, Jan-
uary 2009.
[78] Renewing Communities. Pittsburgh People & Places Tour: Stop 4 (August
Wilson House & Freedom Corner), July 2021.

72
[79] Sala Udin, 68 Stories.
[80] Bev Smith, 82 Stories.
[81] Boswell, Robert - April 22, 1996 | Digital Pitt.
[82] Hughes, John - April 4, 1996 | Digital Pitt.

[83] Laurence Glasco | Department of History | University of Pittsburgh.

[84] Hill District-born jazz great Turrentine dies.
[85] Blender Institute. Textures.

[86] Low Poly Animated People | 3D | Unity Asset Store.

[87] Newsfeed.
[88] Yixiao Fu. Evoking the Post-industrial Landscape Memories through Mixed
Reality Soundscape. thesis, Carnegie Mellon University, March 2021.

[89] AR + GPS Location | | Unity Asset Store.

[90] Unity Technologies. Unity - Manual: Audio files.
[91] Wrong Import Settings are Killing Your Unity Game [Part 2] – The Knights
of Unity.

[92] Unity Technologies. Unity - Scripting API: AudioSource.spatialize.

[93] Unity Technologies. Unity - Manual: Audio Source.
[94] Unity Technologies. Unity - Scripting API: Physics.Raycast.
[95] Unity Technologies. Unity - Scripting API: Renderer.enabled.

[96] Unity Technologies. Unity - Scripting API: AudioSource.clip.

[97] Unity Technologies. Unity - Scripting API: AudioSource.Play.
[98] Button | Unity UI | 1.0.0.

73
A Assets List
B User Test Responses
B.1 Interview Transcripts
Participate 1
The participant has an architectural design background, game design and aug-
mented reality development experiences. He also has computational design
background.
The author: How do you generally feel about this augmented reality walk?
Participant 1: I found it interesting. I think it’s kind of hard to obtain this
experience via other approaches. I can see the people in the scene. It feels like
a game. I mean the immersive feeling is strong. It made me feel that I’m really
viewing that world. Actually I want to ask how to think the location-based
augmented reality works better that the marker based augmented reality?
The author: Generally it works better in two ways. For one thing, the marker
won’t make changes to the physical environment. This is a way to protect the
heritage and also the project can be kept on the site for the longer period since
it doesn’t depend on some physical images or QR codes. Secondly since you
don’t really need to stop and scan anything, the whole experience will be way
more immersive and continuous.
Participant 1: OK I see. I have another question then. The background narra-
tor, do you record this? What are the resources of those audio?
The author: They are oral history from public archives and documentaries.
Participant 1: Oh that works better. That will be definitely more convincing.
The author: As you mentioned that, how well do you think the visual and audio
elements works in this project? How well do they support your understanding?
Participant 1: For me I think the visual parts works better. I mean it’s hard for
you to see the digital views like this. You can go to the museum if you want to
know more about this piece of history, but it’s very hard to gain an immersive
experience like this. The scale of the architecture and people are real. The
textures are real. At least it can make you more impressive. As for the audio, I
think they are related to the visual elements right now. But I think if you can
make it more like a dialog, you know. Like in a game, I can click on a character,
and make him talk to me. I think it will definitely make the whole experience
more interesting.
The author: Oh thank you for your suggestions. The next question will be how
do you think about your current understanding about the district or the local
heritage. Does this experience help? If so, how much does it help?
Participant 1: I think it helped me with the understanding. I mean I’ve been
here in Pittsburgh for a while but I really know nothing about the this location
previously. At least this application made me more interested about this area.

Participate 2

74
Figure 42: Form of assets information, visual part
75
Figure 43: Form of assets information, auditory part
76
The participant has a city planning background, no augmented reality develop-
ment experience.

The author: How do you generally feel about this augmented reality walk?
Participant 2: Generally I think this project is great. It provides a view to see
the process of the urban renewal process and the the city’s development. And it
is shown in the first-person perspective, which is really special. And the views
in the application makes me feel it’s realistic. However, I think it will be better
if there is a brief guidance at the beginning of the whole project. I mean, I was
kind of confused about how it works when I got started.
The author: OK. Then how do you feel about the visual and auditory elements
in this projects? How well did they work to effect your understanding or feel-
ings?
Participant 2: Actually I think the visual elements worked much better than
the auditory aspects. They contributed more because they are more intuitive,
and they looked pretty nice through the augmented reality application.
The author: Great. Do you feel immersive when you went through the whole
journey?
Participant 2: Yes, yes. To be honest, at some moment it was hard for me to
tell the differences between the virtual and physical assets. They looked pretty
real. I think all the assets fit into the background better than I thought.
The author: OK,OK. Then I also want to ask how do you think about your
current understanding about the district or the local heritage. Does this expe-
rience help? If so, how much does it help?
Participant 2: I think it increased my interests in learning about this area. Also
I think it really decreased the time and efforts I need to spend if I want to learn.
For instance, previously I need to read books or articles to learn about it. Or
seeing historical images. And they are black and white mostly(laughing). But
this project better connects the history with current location. It increased my
interests and also make the history more accessible.
The author: Could you please describe a scene that impressed you most?
Participant 2: That’s a good question. Emmm, the first scene jumps to my
mind is the church. It looks pretty nice. But for now I think it’s just a clearing.
The comparison between those two is really impressive.

Participate 3
The participant has backgrounds for architectural design and urban design, no
augmented reality development experience.

The author: How do you feel about this walk generally?

Participant 3: Generally I think it’s nice. But to be honest, at first it was not
very clear to me how the digital assets, I mean those models, I’m not so sure
how they connect to the site. I think not until the third one. I just realized that
this is something happened here previously. And the map, I think it’s kind of
hard to read... At least for me...
The author: Could you please tell me more about the map? Why do you think

77
it’s hard to read?
Participant 3: I think it’s too tiny. I can hardly tell where I was based on it.
The author: OK, I see. Then what do you think about the auditory and visual
elements in this application?
Participant 3: They are pretty cool actually. I found them interesting and they
are reflections of previous history. But I think for one scene...the theater? Yes,
that one, the audio is too long. I basically finished viewing the scene but I had
to stop there for more time to hear the audio. Maybe it will be better if they
can fit more with each other.
The author: Thank you for your opinion! Next question is, how well did those
elements work to effect your understanding or feelings?
Participant 3: I think generally they are good. But I hope to see more details.
If you want to make them more real, you have to provide more details. For now
I don’t think that’s enough.
The author: Sure. Then I also want to ask how do you think about your cur-
rent understanding about the district or the local heritage. Does this experience
help? If so, how much does it help?
Participant 3: To be honest, I didn’t remember too many details. I know there
is a sandwich shop here, a theater, a church, and a performance? Yeah I’m
not so sure if I fully understand all the materials. Sometimes the audio is hard
to understand... I’m not a native speaker (laugh). But I can generally figure
out that they are related to each other. I think they helped me to build the
understanding of this place.
The author: Thank you. Do you have any other comments or suggestions?
Participant 3: Emmm, I think it’s better to add more details still. If you can
make the image move. Animation, yes. I bet it will be much more interesting
if I can see this.

Participate 4
The participant has backgrounds for architectural design and urban planning,
no augmented reality development experience.

The author: Hi, my first question is how do you generally feel about this aug-
mented reality walk?
Participant 4: I like it very much! I found that it’s a nice way for me to learn
more about the history.
The author: Then how do you feel about the visual and auditory elements?
Participant 4: You mean the images? Oh models as well. Emmm...generally I
think they were nice to reflect the historical scenes. I think the images are very
intuitive. They made me feel more about how people used to work and live here
previously. The author: What about the auditory ones? How do you feel about
them?
Participant 4: I think it works as a background instruction. I mean it’s kind of
hard for me to capture all the details but it provided more information for sure.
But not as intuitive as visual elements.
The author: OK I see! Next may I ask how do you think about the immersive

78
level of this experience?
Participant 4: It’s pretty nice. I mean you can walk and see what happened
previously. It’s pretty cool, right? I think the size of the architectures... Yes, the
digital ones. I think they are pretty real. They really fit into the environment,
which made me really surprised. I think it feels pretty real to walk through all
the virtual streets and people. Yes, I think it’s pretty immersive for me.
The author: Thank you. I also want to know that how this experience works
for your understanding about the district?
Participant 4: It helped, of course. I mean I had little knowledge about this
district previously. But I know this used to be an important neighborhood for
the locals and it got removed. People were not happy about this and fought
against it. It’s pretty clear to see through this experience!
The author: Cool. Then do you have any other comments or suggestions?
Participant 4: Emmm, maybe if you can add more scenes in this application, it
will be better.

Participate 5
The participant has backgrounds for architectural design and urban planning,
no augmented reality development experience.

The author: Thank you for having this interview. My first question is how do
you feel about this augmented reality walk generally?
Participant 5: I think it’s great to combining different kinds of assets together
like images, models and sounds. I really like the scene with some musicians in
it... Yes the jazz club. Once the background music started, it made me feel
pretty real immediately.
The author: Oh I see. Then how do you feel you about visual and auditory
elements specifically in this project? Did they support your understanding of
this district?
Participant 5: Emmm, as I said, generally I think they are nice. The audios
worked well. I learned a lot from the narrator in the background. As for the
digital models, they are not bad. But I think they looked not that real. They
still looked like models.
The author: Could you please tell me more about why you felt like this?
Participant 5: I think it’s their textures. You know when we normally render a
model, the texture assets really matter. So I just wonder if you can modify this
issue by changing the textures or rendering materials you choose.
The author: Thank you for this suggestion! Then do you have any other com-
ments or opinions about this project?
Participant 5: Oh one more thing I want to mention, I found the shadow of
the architecture is not right... Yeah, the direction. So are they pre-designed or
pre-set?
The author: Yes, the light and shadow are pre-designed in the game engine.
Participant 5: Yeah, it makes sense. But it will be more real if the shadow can
change based on the real sunlight directions. I guess it’s another reason I found
the models looked not like real architectures.

79
Participate 6
The participant has backgrounds an architectural design background and aug-
mented reality development experiences. He also has computational design
background.

The author: So generally how do you think about this experience?

Participant 6: I like it. I think it successfully offered a method to experience
what happened here previously. I mean in the urban space, it’s very creative
and interesting. The augmented reality worked pretty stable, which was also
nice.
The author: Thank you. How do you feel about the auditory and visual ele-
ments in this project then? Do they help with your understanding about this
district?
Participant 6: Yes I think they helped. Previously I didn’t know much about
the district. But when I saw the images of people who used to live here, I
started to realized here might be a living neighborhood before. I tried to figure
out the reason why they disappeared in the following process. It’s pretty nice,
the exploration based on all the oral history and visual cues.
The author: Good to hear that. Could you please describe a scene that inter-
ested you most or made you impressed?
Participant 6: Emmm. let me see. I think it might be the theater. I saw the
sandwich shop nearby and some staffs. And I hear this part, the story also
in the audio in the background. It’s very interesting to find they are actually
related!
The author: Thank you. Then do you have any other comments about this
walk?
Participant 6: I think it’s necessary to try to add more interactions. Sometimes
I can’t help clicking different elements. If I can get some reactions it will defi-
nitely be more attractive to me.

Participate 7
The participant has backgrounds an architectural design background and com-
putational design background.

The author: Hi, so generally how do you think about this augmented reality
walk? Participant 7: It’s interesting to see a project like this. Especially you
managed to combine models and images together. It’s interesting to walk and
see different things appear on the screen.
The author: Cool. Then how do you feel about those visual and auditory ele-
ments specifically? Like the images, models and the sounds.
Participant 7: Actually I think the 3D model worked well. Real size, real ma-
terial, you know what I mean. May I ask a question actually? Yeah, why you
choose to use images to show human activities? I mean, they didn’t fit well
from my point of view.
The author: I tried to use human models for this part. But the low-poly models

80
are not very ideal to represent real residents in this district.
Participant 7: So for the reality. It makes sense. But I’m still willing to see
a 3D version (laugh). You can add more animations or interactions based on
that.
The author: Thank you for sharing this. Next could you please tell me how
does this experience work for building your understanding about this area?
Participant 7: Oh sure. I think it helped a lot. I mean, even though it’s hard
for me to understand everything, I can still know about the general developing
process of this district. It also triggered my interests to explore more about it in
the future. I think the interaction process is also pretty direct and intuitive. I
have really enjoyed it. What was particularly interesting to me is that there was
a thing there is several workers involved in the sandwich shop and I think this
thing is really good because is there a lifelike and there’s this sort of tenderness
in it and it feels very relatable. I really like this project.
The author: Thank you. So do you have any other suggestions or comments
about this project?
Participant 7: Emmm, maybe not. I think you did a great job! Oh maybe the
3D human figure display I mentioned. I’m willing to see them in the future.

Participate 8
The participant has an architectural design background, game design and aug-
mented reality development experiences.

The author: Thank you for have the walk and the interview. So my first ques-
tion is going to be how you feel about this walk generally?
Participant 8: I really like the project! I think you put many real historical
materials here, like images and the audios. Did you build the models on your
own?
The author: Yes, they were built based on some historical maps. And I also
used some textures from real historical photos.
Participants 8: Oh really cool. I think those elements looked really nice.
The author: So how you feel about those auditory and visual elements specifi-
cally? Did they help with your understanding about this district?
Participant 8: Yes of course. As I said, I liked the visual elements since they are
nicely built and positioned on the location. However, I have to say I learnt more
from the audio. I mean the speaker in the background. Yes the oral history.
The oral history provided more details. Otherwise I couldn’t fully understand-
ing what happened here.
The author: I see. Then could you please describe a scene you felt the most
impressive in this walk?
Participant 8: I think it might be the church. It’s great and impressive. I felt
sorry it was removed from the neighborhood. I can feel that it must mean a lot
for the local people.
The author: Thank you for sharing this. So do you have more suggestions or
options?
Participant 8: Let me see. I think I really hope to see more interactions. I mean

81
you have a very good start here. But I think you can add more details in all the
scenes to make it more interesting and realistic. Like the window or door can
be open. Or I can click something to play a specific human animation. I know
augmented reality platform has this potential.

Participate 9
The participant has backgrounds an architectural design background, compu-
tational design background.

The author: Hi, thank you for have the walk and the interview. First, I want
to hear about how you feel about this walk generally?
Participant 9: I think this is a great project. So all the assets are placed on
their previous location in the history right?
The author: Yes, exactly.
Participant 9: That’s pretty cool. I think it’s kind of magical to see what
happened previously and what it looks like now at the same time on the same
location. I think it greatly help me understand the history and historical ac-
tivities in this site. I like the image showing what the protesters do to protect
their community. I can’t help think if we can also put some architectural design
projects on the location in this way. It will be more intuitive than just seeing
some images.
The author: That’s an interesting idea. May I ask how you feel about the visual
and auditory elements? How did they work for your understanding about this
district?
Participant 9: I think it helped. I mean previously I might choose to see a doc-
umentary if I want to know more about this piece of history. But this project
it provided more. Like the multi-media experience, the interaction. I think I’m
more focused during this process. But I don’t know much about the relationship
among heritage sites. If some visual cues can be offered or there is a correla-
tion among the digital contents of different sites. In the AR environment, I feel
like I just encounter them when walking around and someone introducing the
heritage by showing something. I won’t pay much attention to understanding
the relationship between them. If there are some interactions of feedback, users
might have more motivation to try to understand that.
The author: Thank you. Then could you please describe a scene that interested
you the most or make you very impressive?
Participant 9: I think I will choose the last one. Yes, the protest. It’s because
I think this scene combines the background sounds well with, you know, the
visual parts. The voice of people during the protest, really impressive. And I
can see the Freedom Corner from this viewpoint. It’s very well designed.
The author: Cool. Do you have any more comments or opinion?
Participant 9: Not much except the interaction one. I think it can make the
walk more immersive and interesting.

Participate 10
The participant has backgrounds a design background, no augmented reality

82
development experience.

The author: Hi, thank you for being here. I firstly want to ask how you gener-
ally feel about this experience?
Participant 10: I like it and I think it’s a very creative way to show people
the changes of the urban space during a historical period. It feels like a time
travelling experience (laugh). Yeah, I like it.
The author: Thank you for saying this. Then could you tell me more about how
you specifically feel about those visual and auditory elements in this project?
Participant 10: Visual and auditory... Emmm, let me see. I think I might prefer
visual parts more. Since you can hear the audio from other resources like some
documentaries. But the visual elements, especially the models, they are unique
in this experience. It’s hard to experience the same through other methods.
But I also agree that it’s essential to combine the visual with auditory ones.
The storytelling will be more comprehensive in this way.
The author: I see what you mean. Then how do you think they help with your
understanding about this district?
Participant 10: I think it supported the understanding definitely. I didn’t know
much about this neighborhood before but I think I can rough tell people what
happened here by having this walking. It’s very impressive.
The author: Thank you for saying this. Finally do you have any comments or
instructions?
Participant 10: I think it will be better if we can see more scenes. I think for
now they are separated spots. But if we can see a street, you know, instead of
just some isolated buildings, it will be more immersive and interesting.

B.2 Take Home Survey Responses

Participate 1
Date: 2022-11-27
1. How well do you think the visual elements help with your under-
standing about the heritage?
Extremely helpful.
2. How well do you think the audio elements(environment audios and
oral histories) help with your understanding about the heritage?
Extremely helpful
3. How well do you understand the historical time period shown in
the application?
Clearly understand(more than 50% but not 100%)
4. How do you feel about the interaction process of this augmented
reality application?
It is a new experience for me to know about the history, very interesting and
look forward to seeing more interactions.
5. What do you think about the correlation among heritage sites?
What do you think about the correlation among heritage sites?

83
Participate 2
Date: 2022-11-30
1. How well do you think the visual elements help with your under-
standing about the heritage?
Extremely helpful
2. How well do you think the audio elements(environment audios and
oral histories) help with your understanding about the heritage?
Extremely helpful
3. How well do you understand the historical time period shown in
the application?
Clearly understand(more than 50% but not 100%)
4. How do you feel about the interaction process of this augmented
reality application?
Fairly interactive. Suggest adding a real-time location function to make it even
more interactive.
5. What do you think about the correlation among heritage sites?
Highly correlated.

Participate 3
Date: 2022-11-30
1. How well do you think the visual elements help with your under-
standing about the heritage?
Very helpful
2. How well do you think the audio elements(environment audios and
oral histories) help with your understanding about the heritage?
Neutral
3. How well do you understand the historical time period shown in
the application?
Clearly understand(more than 50% but not 100%)
4. How do you feel about the interaction process of this augmented
reality application?
It would be better with some introduction before the application. I feel confused
in the beginning.
5. What do you think about the correlation among heritage sites?
N/A

Participate 4
Date: 2022-12-1
1. How well do you think the visual elements help with your under-
standing about the heritage?
Very helpful
2. How well do you think the audio elements(environment audios and
oral histories) help with your understanding about the heritage?
Very helpful
3. How well do you understand the historical time period shown in

84
the application?
Neutral
4. How do you feel about the interaction process of this augmented
reality application?
Helpful but somewhat detached from the modern urban context.
5. What do you think about the correlation among heritage sites?
N/A

Participate 5
Date: 2022-12-1
1. How well do you think the visual elements help with your under-
standing about the heritage?
Very helpful
2. How well do you think the audio elements(environment audios and
oral histories) help with your understanding about the heritage?
Extremely helpful
3. How well do you understand the historical time period shown in
the application?
Neutral
4. How do you feel about the interaction process of this augmented
reality application?
I think it greatly help me understand the history and historical activities in this
site. I like the image showing what the protesters do to protect their community.
5. What do you think about the correlation among heritage sites?
Different activities happened in these sites. And these activities combined with
the process of the gentrification in the city. People nowadays can understand
what it was before the gentrification and how people reacted to the gentrifica-
tion.

Participate 6
Date: 2022-12-1
1. How well do you think the visual elements help with your under-
standing about the heritage?
Very helpful
2. How well do you think the audio elements(environment audios and
oral histories) help with your understanding about the heritage?
Very helpful
3. How well do you understand the historical time period shown in
the application?
Totally understand
4. How do you feel about the interaction process of this augmented
reality application?
I think the interaction process is pretty direct and intuitive. I have really en-
joyed it. What was particularly interesting to me is that there was a thing there
is one mom and multiple kids involved and I think this thing is really good
because is there a lifelike and there’s this sort of tenderness in it and it feels

85
very relatable. I really like this project.
5. What do you think about the correlation among heritage sites?
I think the correlation is very strong, because these are all about like black
community and also multi culture community. And I think it’s about different
facet of those community. Like there is recreational, use and residential use and
also commercial use.

Participate 7
Date: 2022-12-2
1. How well do you think the visual elements help with your under-
standing about the heritage?
Very helpful
2. How well do you think the audio elements(environment audios and
oral histories) help with your understanding about the heritage?
Very helpful
3. How well do you understand the historical time period shown in
the application?
Clearly understand(more than 50% but not 100%)
4. How do you feel about the interaction process of this augmented
reality application?
Smooth and interesting.
5. What do you think about the correlation among heritage sites?
I can feel the correlation. But it can be stronger if any maps or historical nar-
rative are provided.

Participate 8
Date: 2022-12-2
1. How well do you think the visual elements help with your under-
standing about the heritage?
Extremely helpful
2. How well do you think the audio elements(environment audios and
oral histories) help with your understanding about the heritage?
Extremely helpful
3. How well do you understand the historical time period shown in
the application?
Clearly understand(more than 50% but not 100%)
4. How do you feel about the interaction process of this augmented
reality application?
The whole is pretty good. It can present intuitive pictures and scenes. The
sound and explanation can also make people understand the stories that hap-
pened before. It can make people understand what they want to show in a
short time. The multi-sensory experience can also make people leave a deeper
impression.
5. What do you think about the correlation among heritage sites?
If there is a connection between several narrators, or if there is a series of ex-
planations between scenes, before and after scenes, maybe the correlation will

86
be better?

Participate 9
Date: 2022-12-3
1. How well do you think the visual elements help with your under-
standing about the heritage?
Very helpful
2. How well do you think the audio elements(environment audios and
oral histories) help with your understanding about the heritage?
Very helpful
3. How well do you understand the historical time period shown in
the application?
Clearly understand(more than 50% but not 100%)
4. How do you feel about the interaction process of this augmented
reality application?
The AR application helps me understand the physical heritage better. It in-
tegrates different forms of media, providing a sense of immersion. But I think
there can be more interaction mechanics to strengthen the user’s participation,
like how the users trigger the oral histories. It would be more impressive if the
digital content can respond to users’ behavior.
5. What do you think about the correlation among heritage sites?
I don’t know much about the relationship among heritage sites. If some visual
cues can be offered or there is a correlation among the digital contents of differ-
ent sites. In the AR environment, I feel like I just encounter them when walking
around and someone introducing the heritage by showing something. I won’t
pay much attention to understanding the relationship between them. If there
are some interactions of feedback, users might have more motivation to try to
understand that.

Participate 9
Date: 2022-12-3
1. How well do you think the visual elements help with your under-
standing about the heritage?
Very helpful
2. How well do you think the audio elements(environment audios and
oral histories) help with your understanding about the heritage?
Very helpful
3. How well do you understand the historical time period shown in
the application?
Clearly understand(more than 50% but not 100%)
4. How do you feel about the interaction process of this augmented
reality application?
I think it’s interesting and engaging. Hope to see more though.
5. What do you think about the correlation among heritage sites?
I’m not very clear about this. But I know they are all about this community in
the past.

87