Semantic Search on the Public Web with Creative Commons

Semantic Search on the Public Web with Creative Commons 2006.03.07 Mike Linksvayer

Billion$ (0) Let's get the hype out of the way....

Billion$ (3) This calls for a mashup...

Billion$ (5) Fortunately CC's founders thought of that from the beginning...

Core Licensing Suite: Creator/Licensor chooses license options NonCommercial No Derivatives ShareAlike Every Creative Commons licenses allows the world to copy and distribute a work provided that the licensee credits the creator/licensor In addition creator/licensor may apply the following conditions:

Internet Archive Free Hosting for CC works http://www.archive.org/

Creative Commons Metadata Example <rdf:RDF xmlns="http://web.resource.org/cc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <Work rdf:about="http://example.com/article.html"> <dc:title>An Example Article</dc:title> <dc:date>2003-10-01</dc:date> <dc:type rdf:resource="http://purl.org/dc/dcmitype/Text" /> <license rdf:resource="http://creativecommons.org/licenses/by-nc-sa/2.5/" /> </Work> <License rdf:about="http://creativecommons.org/licenses/by-nc-sa/2.5/"> <permits rdf:resource="http://web.resource.org/cc/Reproduction" /> <permits rdf:resource="http://web.resource.org/cc/Distribution" /> <requires rdf:resource="http://web.resource.org/cc/Notice" /> <requires rdf:resource="http://web.resource.org/cc/Attribution" /> <prohibits rdf:resource="http://web.resource.org/cc/CommercialUse" /> <permits rdf:resource="http://web.resource.org/cc/DerivativeWorks" /> <requires rdf:resource="http://web.resource.org/cc/ShareAlike" /> </License> </rdf:RDF>

Rights Description Use Cases Discovery Expression Commerce Management(1)

Rights Description vs. Rights Management(2) Copy/Use promotion vs. Copy/Use protection Encourage fans vs. Discourage casual pirates Resource management vs. Customer management Web content model vs. 20 th century content model Not mutually exclusive in theory.

Why Semantic Web? Small organization, no central registration for every license Decentralization: Let a thousand search engines bloom; web as API Existing RDF tools could take advantage of CC RDF

Why RDF-in-HTML comments? (yuck) Considered: Robots.txt-like HTML meta tags LINK to external RDF file RDF-in-HTML comments wins because Metadata colocated with human visible HTML, only single copy & paste for licensors Full power of RDF

CC Search History I Postgresql/tsearch2/python prototype (early 2004) Sloooowwwww, but did what a prototype should do

CC Search History II CC-Nutch (late 2004) Nutch aims to be open source search engine comparable to commercial web scale search engines Built on top of Lucene full text index CC plugin only ~500 lines of code (not counting UI, CC-required additions to Nutch core) http://search.creativecommons.org uses Nutch, >1m CC-licensed pages indexed

CC Search History III Yahoo! Search for Creative Commons (early 2005) Search CC-licensed subset of Yahoo!’s index (~15m* pages) *very rough guesstimate

CC Search History IV Google CC search (November 2005) Search CC-licensed subset of Google’s index (~45m* pages) *very rough guesstimate

CC Search History V (the future) Better metadata formats Image and Video search Derivatives search Content commerce search “ Live” web search “ Management” (desktop, workgroup) Semantic mashups

Future CC metadata formats “ Semantic XHTML” AKA “lowercase semantic web” AKA “microformats” (now) <a rel=“license” href=“ http://creativecommons.org/licenses/by/2.5/ ”> RDF/A AKA XHTML2 metadata (in working group) GRDDL (gleaning resource descriptions from dialects of languages)

Image and Video search Better metadata formats Image and Video search Derivatives search Content commerce search “ Live” web search “ Management” (desktop, workgroup) Semantic mashups

Searching for Derivative Works

Derivatives search RDF/XML snippet: <dc:source rdf:resource=”http://ccmixter.org/media/files/victor/3385”/> Query like Yahoo! link: search or Technorati Cosmos search source:http://ccmixter.org/media/files/victor/3385 “ Who sampled this” as the new “who linked to this”

Content commerce search Transaction costs should be low even if rights are reserved Commercial terms and other commerce described by metadata associated with a work Find me work I can use at a price I can pay for usage rights warranty/paper trail (even if rights not reserved) Reintermediate consumer and creator

“ Live” web search (feeds) Feeds are explicitly metadata-rich (unlike typical web page) Existing blog search ignores metadata Web search will become more like blog search, vice versa?

“ Management” (desktop, workgroup) Desktop search (OS-level) Content creation and media player integration XMP Semantic Wikis

Issues for Semantic Search on the Public Web Metadata quality Trust Scalability Usability Compatibility Critical mass State of the art IR works very well – high expectations!

Semantic Search on the Public Web with Creative Commons 2006.03.07 Mike Linksvayer Questions, feedback, flames: [email_address] http://developer.creativecommons.org

Semantic Search on the Public Web with Creative Commons

More Related Content

Viewers also liked

Similar to Semantic Search on the Public Web with Creative Commons

More from Mike Linksvayer

Recently uploaded

Semantic Search on the Public Web with Creative Commons