Download as PDF, PPTX
![RUN A LUCENE QUERY BUT GET JPA
MANAGED RESULTS
String[] productFields = { "title", "actors.name" };
org.apache.lucene.search.Query luceneQuery = // ...
FullTextEntityManager ftEm =
Search.getFullTextEntityManager( entityManager );
FullTextQuery query = // extends javax.persistence.Query
ftEm.createFullTextQuery( luceneQuery, DVD.class );
List dvds = // Managed entities!
query.setMaxResults(100).getResultList();
int totalNbrOfResults = query.getResultSize();](https://image.slidesharecdn.com/apacheluceneforjavaeedevelopers-150723151345-lva1-app6891/85/Apache-Lucene-for-Java-EE-Developers-33-320.jpg)
![RUN A LUCENE QUERY BUT GET JPA
MANAGED RESULTS
String[] productFields = { "title", "actors.name" };
org.apache.lucene.search.Query luceneQuery = // ...
FullTextEntityManager ftEm =
Search.getFullTextEntityManager( entityManager );
FullTextQuery query = // extends javax.persistence.Query
ftEm.createFullTextQuery( luceneQuery, DVD.class );
List dvds = // Managed entities!
query.setMaxResults(100).getResultList();
int totalNbrOfResults = query.getResultSize();](https://image.slidesharecdn.com/apacheluceneforjavaeedevelopers-150723151345-lva1-app6891/75/Apache-Lucene-for-Java-EE-Developers-33-2048.jpg)
Uploaded byVirtual JBoss User Group
Apache Lucene for Java EE Developers
The document provides a detailed introduction to Apache Lucene for Java EE developers, highlighting its integration with Hibernate and Infinispan for search functionality in applications. It covers key concepts such as search engine requirements, index management, and how to effectively implement Lucene within a JPA context. Additionally, it outlines future plans and improvements for both Infinispan and Hibernate Search, emphasizing the importance of efficient search capabilities in modern applications.
More Related Content
![Faster java ee builds with gradle [con4921]](https://cdn.slidesharecdn.com/ss_thumbnails/fasterjavaeebuildswithgradlecon4921-160927042155-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Similar to Apache Lucene for Java EE Developers
Apache Lucene for Java EE Developers
- 1. APACHE LUCENE FOR JAVAEEDEVELOPERS VIRTUAL:JBUG by @SanneGrinovero
- 2. A QUICK INTRODUCTION HIBERNATETEAM Hibernate Search project lead Hibernate OGM team, occasionally Hibernate ORM
- 3. INFINISPAN TEAM The Luceneguy: Infinispan Query, Infinispan Lucene Directory
- 4. OTHER PROJECTS IHELP WITH... WildFly, JGroups, Apache Lucene, ...
- 5. SUPPORTED BY Red Hatand a lot of passion for OSS
- 6. AGENDA What is andhow can it help you Integrations with a JPA application via How does this all relate with and Lucene index management Plans and wishlist for the future Apache Lucene Hibernate Search Infinispan WildFly
- 7. THE SEARCH PROBLEM Hello,I'm looking for a book in your online shop having primary key #2342
- 9. SQL CAN HANDLETEXT The LIKE operator?
- 10. LET'S REFRESH SOMEHISTORY ON THE WIKIPEDIA Select * from WikipediaPages p where p.content LIKE ?; Select * from WikipediaPages p where p.title LIKE ?; Select * from WikipediaPages p where (lowercase(p.content) LIKE %:1% OR lowercase(p.content) LIKE %:2% OR lowercase(p.content) LIKE %:3% OR ...);
- 14. AM I CHEATING? I'mquoting some very successfull web companies. How many can you list which do not provide an effective search engine? Why is that?
- 15. REQUIREMENTS FOR ASEARCH ENGINE Need to guess what you want w/o you typing all of the content We all hate forms We want the results in the blink of an eye We want the right result on top: Relevance
- 17. SOME MORE THINGSTO CONSIDER: Approximate word matches Stemming / Language specific analysis Typos Synonyms, Abbreviations, Technical Language specializations
- 18. BASICS: KEYWORD EXTRACTION Onhow to improve running by Scott 1. Tokenization & Analysis: how improv run scott 2. Scoring
- 19. APACHE LUCENE Open sourceApache™ top level project Primarily Java, ported to many other languages and platforms Extremely popular, it's everywhere! High pace of improvement, excellent team Most impressive testing
- 20. AS A JAVAEEDEVELOPER: You are familiar with JPA But Lucene is much better than a relational database to address this problem Easy integration with the platform is a requirement
- 22. LET'S INTRODUCE APACHELUCENE VIA HIBERNATE SEARCH Deeply but transparently integrated with Hibernate's
- 23. EntityManager Internally uses advancedApache Lucene features, but protects your deadlines from the lower level details Gets great performance out of it Simple annotations, yet many flexible override options
- 24. Does not preventyou to perform any form of advanced / native Lucene query
- 25. Transparent index statesynchronization Transaction integrations Options to rebuild the index efficiently Failover and clustering integration points Flexible Error handling
- 26.
- 27. HOW TO INDEXA TRIVIAL DOMAIN MODEL @Entity public class Actor { @Id Integer id; String name; }
- 28. @Indexed @Entity public class Actor { @Id Integer id; String name; }
- 29. @Indexed @Entity public class Actor { @Id Integer id; @Field String name; }
- 30. LET'S INTRODUCE RELATIONS @Entity public class DVD { @Id Integer id; String title; @ManyToMany Set<Actor> actors = new HashSet<>(); }
- 31. @Indexed @Entity public class DVD { @Id Integer id; @Field String title; @ManyToMany @IndexedEmbedded Set<Actor> actors = new HashSet<>(); }
- 32. INDEX FIELDS FORACTOR id name 1 Harrison Ford 2 Kirsten Dunst INDEX FIELDS FOR DVD id title actors.name * 1 Melancholia {Kirsten Dunst, Charlotte Gainsbourg, Kiefer Sutherland} 2 The Force Awakens {Harrison Ford, Mark Hamill, Carrie Fisher}
- 33. RUN A LUCENEQUERY BUT GET JPA MANAGED RESULTS String[] productFields = { "title", "actors.name" }; org.apache.lucene.search.Query luceneQuery = // ... FullTextEntityManager ftEm = Search.getFullTextEntityManager( entityManager ); FullTextQuery query = // extends javax.persistence.Query ftEm.createFullTextQuery( luceneQuery, DVD.class ); List dvds = // Managed entities! query.setMaxResults(100).getResultList(); int totalNbrOfResults = query.getResultSize();
- 34.
- 35. LUCENE & TEXTANALYSIS @Indexed(index = "tweets") @Analyzer(definition = "english") @AnalyzerDef(name = "english", tokenizer = @TokenizerDef( factory = StandardTokenizerFactory.class), filters = { @TokenFilterDef(factory = ASCIIFoldingFilterFactory.class), @TokenFilterDef(factory = LowerCaseFilterFactory.class), @TokenFilterDef(factory = StopFilterFactory.class, params = { @Parameter(name = "words", value = "stoplist.properties"), @Parameter(name = "ignoreCase", value = "false") }) }) @Entity public class Tweet {
- 36.
- 37.
- 40.
- 41. "MORE LIKE THIS" Coffee decaffInstance = ... // you already have one QueryBuilder qb = getCoffeeQueryBuilder(); Query mltQuery = qb .moreLikeThis() .comparingAllFields() .toEntityWithId( decaffInstance.getId() ) .createQuery(); List results = fullTextEntityManager .createFullTextQuery( mltQuery, Coffee.class ) .list();
- 42.
- 43. HOW TO RUNTHIS ON WILDFLY?
- 44. ARCHITECTURE & INDEXMANAGEMENT ah, the catch!
- 45. Indexes need tobe stored, updated and read from. You can have many indexes, managed independently An index can be written by an exclusive writer only Backends can be configured differently per index Index storage - the Directory - can also be configured per index
- 48. THE INFINISPAN /LUCENE INTEGRATIONS
- 49. WHAT IS INFINISPAN? InMemory Key/Value Store ASL v2 License Scalable JTA Transactions Persistence (File/JDBC/LevelDB/...) Local/Clustered Embedded/Server ...
- 50. INFINISPAN / JAVAEE? JavaEE:JCache implementation A core component of WildFly "Embedded" mode does not depend on WildFly Hibernate 2n level cache
- 51. INFINISPAN / APACHELUCENE? Lucene integrations for Querying the datagrid! Lucene integrations to store the index! Hibernate Search integrations!
- 52. HIBERNATE SEARCH &INFINISPAN QUERY Same underlying technology Same API to learn Same indexing configuration options
- 54. Same annotations, notan entity: @Indexed @AnalyzerDef(name = "lowercaseKeyword", tokenizer = @TokenizerDef(factory = KeywordTokenizerFactory.class), filters = {@TokenFilterDef(factory = LowerCaseFilterFactory.class)} ) @SerializeWith(CountryExternalizer.class) public class Country { @Field(store = Store.YES) @Analyzer(definition = "lowercaseKeyword") private String name;
- 55. Store the JavaPOJO classes in the cache directly: Country uk = ... cache.put("UK", uk );
- 56. USING LUCENE QUERYPARSER QueryParser qp = new QueryParser("default", new StandardAnalyzer()); Query luceneQ = qp .parse("+station.name:airport +year:2014 +month:12 +(avgTemp < 0)"); CacheQuery cq = Search.getSearchManager(cache) .getQuery(luceneQ, DaySummary.class); List<Object> results = query.list();
- 57.
- 58.
- 59. GETTING STARTED WITHINFINISPAN <dependency> <groupId>org.infinispan</groupId> <artifactId>infinispanembedded</artifactId> <version>7.2.3.Final</version> </dependency> EmbeddedCacheManager cacheManager = new DefaultCacheManager(); Cache<String,String> cache = cacheManager.getCache(); cache.put("key", "data goes here");
- 60.
- 61.
- 62.
- 63.
- 64. QUERYING Apache Lucene Index NativeMap Reduce Index-less Hadoop and Spark (coming)
- 65.
- 66.
- 67. QUERY - RAMSTORAGE Configuration configuration = new ConfigurationBuilder() .indexing().index(Index.LOCAL) .addProperty("default.worker.execution", "async") .addProperty("default.directory_provider", "ram") .build(); EmbeddedCacheManager cm = new DefaultCacheManager(configuration); Cache<Integer, DaySummary> cache = cm.getCache();
- 68. QUERY - INFINISPANSTORAGE Configuration configuration = new ConfigurationBuilder() .indexing().index(Index.LOCAL) .addProperty("default.worker.execution", "async") .addProperty("default.directory_provider", "infinispan") .build(); EmbeddedCacheManager cm = new DefaultCacheManager(configuration); Cache<Integer, DaySummary> cache = cm.getCache();
- 69. QUERY - FILESYSTEMSTORAGE Configuration configuration = new ConfigurationBuilder() .indexing().index(Index.LOCAL) .addProperty("default.directory_provider", "filesystem") .addProperty("default.indexBase", "/path/to/index); .build();
- 70. QUERY - INFINISPANINDEXMANAGER Configuration configuration = new ConfigurationBuilder() .indexing().index(Index.LOCAL) .addProperty("default.worker.execution", "async") .addProperty("default.indexmanager", "org.infinispan.query.indexmanager.InfinispanIndexManager" .build(); EmbeddedCacheManager cm = new DefaultCacheManager(configuration); Cache<Integer, DaySummary> cache = cm.getCache();
- 72. THE INFINISPAN LUCENEDIRECTORY Storing the Apache Lucene index in an high-performance in memory data grid
- 73. You don't needHibernate Search to cluster your existing Lucene application, but you'll need some external coordination to guarantee the single IndexWriter.
- 74. INFINISPAN QUERY ANDTHE LUCENE DIRECTORY IN ACTION Weather Demo by Gustavo Nalle Fernandes DEMO Indexed NOAA.gov data from 1901 to 2014 ~10M summaries Yearly country max recorded temperature by month Cache<Integer, DaySummary>
- 75. WHAT'S ON THEHORIZON FOR INFINISPAN Improvements in indexing performance Hadoop and Spark integration experiments Combining indexed & non-indexed query capabilities, including remote queries
- 76. WHAT'S COMING FORHIBERNATE SEARCH Upgrading to Lucene 5 Experiment integrations with REST based Lucene servers (Solr, ElasticSearch) Improved backends to simplify clustering setup GSOC: generic JPA support and improved developer tooling A lot more! See also the roadmap
- 77. THANK YOU! Some references: ,the super simple , the The website, the by The website Our team's blog (requires Chrome) Apache Lucene website Hibernate Search website Hibernate Search JPA demo WildFly integration tests Infinispan Weather Demo @gustavonalle WildFly in.relation.to Export these slides to PDF