Author: echatellier Date: 2011-06-22 11:47:11 +0200 (Wed, 22 Jun 2011) New Revision: 959 Url: http://nuiton.org/repositories/revision/wikitty/959 Log: Add new wikitty-lucene module Added: trunk/wikitty-lucene/ trunk/wikitty-lucene/LICENSE.txt trunk/wikitty-lucene/README.txt trunk/wikitty-lucene/changelog.txt trunk/wikitty-lucene/pom.xml trunk/wikitty-lucene/src/ trunk/wikitty-lucene/src/main/ trunk/wikitty-lucene/src/main/java/ trunk/wikitty-lucene/src/main/java/org/ trunk/wikitty-lucene/src/main/java/org/nuiton/ trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/ trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/ trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/ trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/Restriction2Lucene.java trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/WikittyLuceneConstants.java trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/WikittySearchEngineLucene.java trunk/wikitty-lucene/src/main/resources/ trunk/wikitty-lucene/src/test/ trunk/wikitty-lucene/src/test/java/ trunk/wikitty-lucene/src/test/java/org/ trunk/wikitty-lucene/src/test/java/org/nuiton/ trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/ trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/ trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/ trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/WikittySearchEngineLuceneTest.java trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/WikittyServiceLucene.java trunk/wikitty-lucene/src/test/resources/ Property changes on: trunk/wikitty-lucene ___________________________________________________________________ Added: svn:ignore + .classpath .project .settings bin target Added: trunk/wikitty-lucene/LICENSE.txt =================================================================== --- trunk/wikitty-lucene/LICENSE.txt (rev 0) +++ trunk/wikitty-lucene/LICENSE.txt 2011-06-22 09:47:11 UTC (rev 959) @@ -0,0 +1,166 @@ + GNU LESSER GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/> + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + + This version of the GNU Lesser General Public License incorporates +the terms and conditions of version 3 of the GNU General Public +License, supplemented by the additional permissions listed below. + + 0. Additional Definitions. + + As used herein, "this License" refers to version 3 of the GNU Lesser +General Public License, and the "GNU GPL" refers to version 3 of the GNU +General Public License. + + "The Library" refers to a covered work governed by this License, +other than an Application or a Combined Work as defined below. + + An "Application" is any work that makes use of an interface provided +by the Library, but which is not otherwise based on the Library. +Defining a subclass of a class defined by the Library is deemed a mode +of using an interface provided by the Library. + + A "Combined Work" is a work produced by combining or linking an +Application with the Library. The particular version of the Library +with which the Combined Work was made is also called the "Linked +Version". + + The "Minimal Corresponding Source" for a Combined Work means the +Corresponding Source for the Combined Work, excluding any source code +for portions of the Combined Work that, considered in isolation, are +based on the Application, and not on the Linked Version. + + The "Corresponding Application Code" for a Combined Work means the +object code and/or source code for the Application, including any data +and utility programs needed for reproducing the Combined Work from the +Application, but excluding the System Libraries of the Combined Work. + + 1. Exception to Section 3 of the GNU GPL. + + You may convey a covered work under sections 3 and 4 of this License +without being bound by section 3 of the GNU GPL. + + 2. Conveying Modified Versions. + + If you modify a copy of the Library, and, in your modifications, a +facility refers to a function or data to be supplied by an Application +that uses the facility (other than as an argument passed when the +facility is invoked), then you may convey a copy of the modified +version: + + a) under this License, provided that you make a good faith effort to + ensure that, in the event an Application does not supply the + function or data, the facility still operates, and performs + whatever part of its purpose remains meaningful, or + + b) under the GNU GPL, with none of the additional permissions of + this License applicable to that copy. + + 3. Object Code Incorporating Material from Library Header Files. + + The object code form of an Application may incorporate material from +a header file that is part of the Library. You may convey such object +code under terms of your choice, provided that, if the incorporated +material is not limited to numerical parameters, data structure +layouts and accessors, or small macros, inline functions and templates +(ten or fewer lines in length), you do both of the following: + + a) Give prominent notice with each copy of the object code that the + Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the object code with a copy of the GNU GPL and this license + document. + + 4. Combined Works. + + You may convey a Combined Work under terms of your choice that, +taken together, effectively do not restrict modification of the +portions of the Library contained in the Combined Work and reverse +engineering for debugging such modifications, if you also do each of +the following: + + a) Give prominent notice with each copy of the Combined Work that + the Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the Combined Work with a copy of the GNU GPL and this license + document. + + c) For a Combined Work that displays copyright notices during + execution, include the copyright notice for the Library among + these notices, as well as a reference directing the user to the + copies of the GNU GPL and this license document. + + d) Do one of the following: + + 0) Convey the Minimal Corresponding Source under the terms of this + License, and the Corresponding Application Code in a form + suitable for, and under terms that permit, the user to + recombine or relink the Application with a modified version of + the Linked Version to produce a modified Combined Work, in the + manner specified by section 6 of the GNU GPL for conveying + Corresponding Source. + + 1) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (a) uses at run time + a copy of the Library already present on the user's computer + system, and (b) will operate properly with a modified version + of the Library that is interface-compatible with the Linked + Version. + + e) Provide Installation Information, but only if you would otherwise + be required to provide such information under section 6 of the + GNU GPL, and only to the extent that such information is + necessary to install and execute a modified version of the + Combined Work produced by recombining or relinking the + Application with a modified version of the Linked Version. (If + you use option 4d0, the Installation Information must accompany + the Minimal Corresponding Source and Corresponding Application + Code. If you use option 4d1, you must provide the Installation + Information in the manner specified by section 6 of the GNU GPL + for conveying Corresponding Source.) + + 5. Combined Libraries. + + You may place library facilities that are a work based on the +Library side by side in a single library together with other library +facilities that are not Applications and are not covered by this +License, and convey such a combined library under terms of your +choice, if you do both of the following: + + a) Accompany the combined library with a copy of the same work based + on the Library, uncombined with any other library facilities, + conveyed under the terms of this License. + + b) Give prominent notice with the combined library that part of it + is a work based on the Library, and explaining where to find the + accompanying uncombined form of the same work. + + 6. Revised Versions of the GNU Lesser General Public License. + + The Free Software Foundation may publish revised and/or new versions +of the GNU Lesser General Public License from time to time. Such new +versions will be similar in spirit to the present version, but may +differ in detail to address new problems or concerns. + + Each version is given a distinguishing version number. If the +Library as you received it specifies that a certain numbered version +of the GNU Lesser General Public License "or any later version" +applies to it, you have the option of following the terms and +conditions either of that published version or of any later version +published by the Free Software Foundation. If the Library as you +received it does not specify a version number of the GNU Lesser +General Public License, you may choose any version of the GNU Lesser +General Public License ever published by the Free Software Foundation. + + If the Library as you received it specifies that a proxy can decide +whether future versions of the GNU Lesser General Public License shall +apply, that proxy's public statement of acceptance of any version is +permanent authorization for you to choose that version for the +Library. + Added: trunk/wikitty-lucene/pom.xml =================================================================== --- trunk/wikitty-lucene/pom.xml (rev 0) +++ trunk/wikitty-lucene/pom.xml 2011-06-22 09:47:11 UTC (rev 959) @@ -0,0 +1,98 @@ +<?xml version="1.0" encoding="UTF-8"?> +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> + + <modelVersion>4.0.0</modelVersion> + + <parent> + <groupId>org.nuiton</groupId> + <artifactId>wikitty</artifactId> + <version>3.1.3-SNAPSHOT</version> + </parent> + + <!-- ************************************************************* --> + <!-- *** POM Relationships *************************************** --> + <!-- ************************************************************* --> + + <groupId>org.nuiton.wikitty</groupId> + <artifactId>wikitty-lucene</artifactId> + + <dependencies> + + <!-- sibling dependencies --> + <dependency> + <groupId>${project.groupId}</groupId> + <artifactId>wikitty-api</artifactId> + <version>${project.version}</version> + </dependency> + <dependency> + <groupId>${project.groupId}</groupId> + <artifactId>wikitty-api</artifactId> + <version>${project.version}</version> + <classifier>tests</classifier> + <scope>test</scope> + </dependency> + + <dependency> + <groupId>jboss.jbossts</groupId> + <artifactId>jbossjta</artifactId> + </dependency> + + <dependency> + <groupId>org.nuiton</groupId> + <artifactId>nuiton-utils</artifactId> + </dependency> + + <!-- Lucene --> + <dependency> + <groupId>org.apache.lucene</groupId> + <artifactId>lucene-core</artifactId> + </dependency> + <dependency> + <groupId>org.apache.lucene</groupId> + <artifactId>lucene-analyzers</artifactId> + </dependency> + + <dependency> + <groupId>com.browseengine</groupId> + <artifactId>bobo-browse</artifactId> + <version>2.5.0</version> + </dependency> + + <dependency> + <groupId>org.slf4j</groupId> + <artifactId>slf4j-log4j12</artifactId> + </dependency> + + <dependency> + <groupId>commons-logging</groupId> + <artifactId>commons-logging</artifactId> + </dependency> + + <dependency> + <groupId>commons-lang</groupId> + <artifactId>commons-lang</artifactId> + </dependency> + + <dependency> + <groupId>commons-collections</groupId> + <artifactId>commons-collections</artifactId> + </dependency> + <!-- TEST --> + <dependency> + <groupId>junit</groupId> + <artifactId>junit</artifactId> + </dependency> + + </dependencies> + + <!-- ************************************************************* --> + <!-- *** Project Information ************************************* --> + <!-- ************************************************************* --> + + <name>Wikitty :: wikitty-lucene</name> + + <description>Wikitty lucene search engine</description> + <inceptionYear>2011</inceptionYear> + +</project> + Added: trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/Restriction2Lucene.java =================================================================== --- trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/Restriction2Lucene.java (rev 0) +++ trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/Restriction2Lucene.java 2011-06-22 09:47:11 UTC (rev 959) @@ -0,0 +1,376 @@ +/* + * #%L + * + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Codelutin, Chatellier Eric + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Lesser Public License for more details. + * + * You should have received a copy of the GNU General Lesser Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/lgpl-3.0.html>. + * #L% + */ + +package org.nuiton.wikitty.storage.lucene; + +import org.apache.commons.lang.NotImplementedException; +import org.apache.lucene.analysis.Analyzer; +import org.apache.lucene.index.Term; +import org.apache.lucene.queryParser.ParseException; +import org.apache.lucene.queryParser.QueryParser; +import org.apache.lucene.search.BooleanClause; +import org.apache.lucene.search.BooleanQuery; +import org.apache.lucene.search.Query; +import org.apache.lucene.search.TermQuery; +import org.nuiton.wikitty.WikittyException; +import org.nuiton.wikitty.search.operators.And; +import org.nuiton.wikitty.search.operators.AssociatedRestriction; +import org.nuiton.wikitty.search.operators.Between; +import org.nuiton.wikitty.search.operators.Contains; +import org.nuiton.wikitty.search.operators.Element; +import org.nuiton.wikitty.search.operators.EndsWith; +import org.nuiton.wikitty.search.operators.Equals; +import org.nuiton.wikitty.search.operators.Greater; +import org.nuiton.wikitty.search.operators.GreaterOrEqual; +import org.nuiton.wikitty.search.operators.In; +import org.nuiton.wikitty.search.operators.Keyword; +import org.nuiton.wikitty.search.operators.Less; +import org.nuiton.wikitty.search.operators.LessOrEqual; +import org.nuiton.wikitty.search.operators.Like; +import org.nuiton.wikitty.search.operators.Not; +import org.nuiton.wikitty.search.operators.NotEquals; +import org.nuiton.wikitty.search.operators.Null; +import org.nuiton.wikitty.search.operators.Or; +import org.nuiton.wikitty.search.operators.Restriction; +import org.nuiton.wikitty.search.operators.StartsWith; +import org.nuiton.wikitty.search.operators.Unlike; + +/** + * Concert wikitty restriction to lucene search query. + * + * @author chatellier + * @version $Revision$ + * + * Last update : $Date$ + * By : $Author$ + */ +public class Restriction2Lucene { + + /** Query parser depending on {@link Analyzer}. */ + protected QueryParser parser; + + public Restriction2Lucene(Analyzer analyzer) { + parser = new QueryParser(WikittySearchEngineLucene.WIKITTY_LUCENE_VERSION, + WikittyLuceneConstants.WIKITTY_LUCENE_DEFAULT_FIELD, analyzer); + } + + /** + * Convert a wikitty restriction to lucene search query. + * + * @param restriction restriction to convert + * @return lucene query + * @throws WikittyException if query can't be parsed + */ + public Query toLucene(Restriction restriction) throws WikittyException { + + Query query = null; + + try { + switch (restriction.getName()) { + case TRUE: + query = true2Lucene(); + break; + case FALSE: + query = false2Lucene(); + break; + case NOT: + query = not2Lucene((Not)restriction); + break; + case AND: + query = and2Lucene((And)restriction); + break; + case OR: + query = or2Lucene((Or)restriction); + break; + case EQUALS: + query = neq2Lucene((Equals)restriction); + break; + case NOT_EQUALS: + query = less2Lucene((NotEquals)restriction); + break; + case LESS: + query = lessEq2Lucene((Less)restriction); + break; + case LESS_OR_EQUAL: + query = lessEq2Lucene((LessOrEqual)restriction); + break; + case GREATER: + query = great2Lucene((Greater)restriction); + break; + case GREATER_OR_EQUAL: + query = greatEq2Lucene((GreaterOrEqual)restriction); + break; + case BETWEEN: + query = between2Lucene((Between)restriction); + break; + case CONTAINS: + query = contains2Lucene((Contains)restriction); + break; + case IN: + query = in2Lucene((In)restriction); + break; + case STARTS_WITH: + query = start2Lucene((StartsWith)restriction); + break; + case ENDS_WITH: + query = end2Lucene((EndsWith)restriction); + break; + case LIKE: + query = like2Lucene((Like)restriction); + break; + case UNLIKE: + query = unlike2Lucene((Unlike)restriction); + break; + case ASSOCIATED: + query = associated2Lucene((AssociatedRestriction)restriction); + break; + case KEYWORD: + query = keyword2Lucene((Keyword)restriction); + break; + case IS_NULL: + query = isNull2Lucene((Null)restriction); + break; + case IS_NOT_NULL: + query = isNotNull2Lucene((Null)restriction); + break; + default: + throw new WikittyException("this kind of restriction is not supported : " + + restriction.getName().toString()); + } + } catch (ParseException ex) { + throw new WikittyException("Can't transform restriction to lucene", ex); + } + return query; + } + + protected String element2Lucene(Element element) { + String result = element.getName(); + //result = fieldModifer.convertToSolr(transaction, result); + return result; + } + + /** + * @param restriction + * @return + */ + protected Query isNotNull2Lucene(Null restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query isNull2Lucene(Null restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + * @throws ParseException + */ + protected Query keyword2Lucene(Keyword restriction) throws ParseException { + String value = restriction.getValue(); + Query query = parser.parse(value); + return query; + } + + /** + * @param restriction + * @return + */ + protected Query associated2Lucene(AssociatedRestriction restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query unlike2Lucene(Unlike restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query like2Lucene(Like restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query end2Lucene(EndsWith restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query start2Lucene(StartsWith restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query in2Lucene(In restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query contains2Lucene(Contains restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query between2Lucene(Between restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query greatEq2Lucene(GreaterOrEqual restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query great2Lucene(Greater restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query lessEq2Lucene(LessOrEqual restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query lessEq2Lucene(Less restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query less2Lucene(NotEquals restriction) { + throw new NotImplementedException("Not yet implemented"); + } + + /** + * @param restriction + * @return + */ + protected Query neq2Lucene(Equals restriction) { + String field = element2Lucene(restriction.getElement()); + Term term = new Term(field, restriction.getValue()); + TermQuery query = new TermQuery(term); + return query; + } + + /** + * @param or + * @return + */ + protected Query or2Lucene(Or or) { + if (or.getRestrictions() == null) { + throw new WikittyException("and.restrictions is null"); + } + if (or.getRestrictions().size() < 2) { + throw new WikittyException("AND is an operator that handle 2 operand at least"); + } + BooleanQuery query = new BooleanQuery(); + for (Restriction restriction : or.getRestrictions()) { + Query subQuery = toLucene(restriction); + query.add(subQuery, BooleanClause.Occur.SHOULD); + } + return query; + } + + /** + * @param and + * @return + */ + protected Query and2Lucene(And and) { + if (and.getRestrictions() == null) { + throw new WikittyException("and.restrictions is null"); + } + if (and.getRestrictions().size() < 2) { + throw new WikittyException("AND is an operator that handle 2 operand at least"); + } + BooleanQuery query = new BooleanQuery(); + for (Restriction restriction : and.getRestrictions()) { + Query subQuery = toLucene(restriction); + query.add(subQuery, BooleanClause.Occur.MUST); + } + return query; + } + + protected Query true2Lucene() { + // *:* + Query query = new TermQuery(new Term("*", "*")); + return query; + } + + protected Query false2Lucene() { + // *:* - *:* + BooleanQuery query = new BooleanQuery(); + query.add(true2Lucene(), BooleanClause.Occur.SHOULD); + query.add(true2Lucene(), BooleanClause.Occur.MUST_NOT); + return query; + } + + protected Query not2Lucene(Not not) { + // *:* - *:* + BooleanQuery query = new BooleanQuery(); + query.add(true2Lucene(), BooleanClause.Occur.SHOULD); + Query subQuery = toLucene(not.getRestriction()); + query.add(subQuery, BooleanClause.Occur.MUST_NOT); + return query; + } +} Property changes on: trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/Restriction2Lucene.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/WikittyLuceneConstants.java =================================================================== --- trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/WikittyLuceneConstants.java (rev 0) +++ trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/WikittyLuceneConstants.java 2011-06-22 09:47:11 UTC (rev 959) @@ -0,0 +1,52 @@ +/* + * #%L + * + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Codelutin, Chatellier Eric + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Lesser Public License for more details. + * + * You should have received a copy of the GNU General Lesser Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/lgpl-3.0.html>. + * #L% + */ + +package org.nuiton.wikitty.storage.lucene; + +/** + * Constants used in wikitty lucene search engine. + * + * @author chatellier + * @version $Revision$ + * + * Last update : $Date$ + * By : $Author$ + */ +public class WikittyLuceneConstants { + + /** + * Query search default field. + */ + static final public String WIKITTY_LUCENE_DEFAULT_FIELD = "text"; + + /** + * Prefix utiliser pour les champs ajouter lors de l'indexation. Ce prefix + * evite d'avoir des conflits entre un nom d'extension et un champs ajoute. + */ + static final public String WIKITTY_LUCENE_PREFIX = "#"; + + /** Id field in lucene. */ + static final public String LUCENE_ID = WIKITTY_LUCENE_PREFIX + "id"; +} Property changes on: trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/WikittyLuceneConstants.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/WikittySearchEngineLucene.java =================================================================== --- trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/WikittySearchEngineLucene.java (rev 0) +++ trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/WikittySearchEngineLucene.java 2011-06-22 09:47:11 UTC (rev 959) @@ -0,0 +1,382 @@ +/* + * #%L + * + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Codelutin, Chatellier Eric + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Lesser Public License for more details. + * + * You should have received a copy of the GNU General Lesser Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/lgpl-3.0.html>. + * #L% + */ + +package org.nuiton.wikitty.storage.lucene; + +import java.io.File; +import java.io.IOException; +import java.lang.reflect.Constructor; +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; + +import org.apache.commons.io.IOUtils; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.lucene.analysis.Analyzer; +import org.apache.lucene.analysis.standard.StandardAnalyzer; +import org.apache.lucene.document.Document; +import org.apache.lucene.document.Field; +import org.apache.lucene.document.Field.Index; +import org.apache.lucene.document.Field.Store; +import org.apache.lucene.index.IndexWriter; +import org.apache.lucene.index.IndexWriterConfig; +import org.apache.lucene.index.Term; +import org.apache.lucene.search.IndexSearcher; +import org.apache.lucene.search.Query; +import org.apache.lucene.search.ScoreDoc; +import org.apache.lucene.search.Sort; +import org.apache.lucene.search.SortField; +import org.apache.lucene.search.TopDocs; +import org.apache.lucene.store.Directory; +import org.apache.lucene.store.NIOFSDirectory; +import org.apache.lucene.util.Version; +import org.nuiton.util.ApplicationConfig; +import org.nuiton.wikitty.WikittyConfigOption; +import org.nuiton.wikitty.WikittyException; +import org.nuiton.wikitty.WikittyUtil; +import org.nuiton.wikitty.entities.Wikitty; +import org.nuiton.wikitty.search.Criteria; +import org.nuiton.wikitty.search.PagedResult; +import org.nuiton.wikitty.search.TreeNodeResult; +import org.nuiton.wikitty.search.operators.Restriction; +import org.nuiton.wikitty.services.WikittyTransaction; +import org.nuiton.wikitty.storage.WikittyExtensionStorage; +import org.nuiton.wikitty.storage.WikittySearchEngine; + +/** + * Wikitty search engine implementation based on Lucene for storage and search + * engine and bobo for facetting support. + * + * @author chatellier + * @version $Revision$ + * + * Last update : $Date$ + * By : $Author$ + */ +public class WikittySearchEngineLucene implements WikittySearchEngine { + + /** Class logger. */ + final static private Log log = LogFactory.getLog(WikittySearchEngineLucene.class); + + /** Index directory. */ + protected Directory indexDirectory; + + /** Reader, writer and query analyzer. */ + protected Analyzer indexAnalyzer; + + /** Lucene version used by wikitty. */ + protected static final Version WIKITTY_LUCENE_VERSION = Version.LUCENE_32; + + /** Index writer config. */ + protected IndexWriterConfig indexWriterConfig; + + /** + * Constructor. + * + * @param config wikitty config + * @param extensionStorage extension storage + */ + public WikittySearchEngineLucene(ApplicationConfig config, WikittyExtensionStorage extensionStorage) { + + try { + indexDirectory = getIndexDirectory(config); + indexAnalyzer = getIndexAnalyzer(config); + indexWriterConfig = new IndexWriterConfig(WIKITTY_LUCENE_VERSION, indexAnalyzer); + } catch (IOException ex) { + throw new WikittyException("Can't init lucene directory", ex); + } + + } + + /** + * Get directory object representing lucene index directory. + * + * @param config application config + * @return index directory + * @throws IOException + */ + protected Directory getIndexDirectory(ApplicationConfig config) throws IOException { + + String luceneDirFactoryKey = WikittyConfigOption.WIKITTY_SEARCHENGINE_LUCENE_INDEX_DIRECTORY.getKey(); + String indexPath = config.getOption(luceneDirFactoryKey); + File indexFile = new File(indexPath); + + if (log.isInfoEnabled()) { + log.info("Using lucene directory : " + indexFile); + } + + Directory directory = new NIOFSDirectory(indexFile); + + return directory; + } + + /** + * Return index analyzer defined in configuration or fallback to + * {@link StandardAnalyzer}. + * + * @param config + * @return + */ + protected Analyzer getIndexAnalyzer(ApplicationConfig config) { + + Analyzer analyzer = null; + + String luceneAnalyzerFactoryKey = WikittyConfigOption.WIKITTY_SEARCHENGINE_LUCENE_INDEX_ANALYZER.getKey(); + + try { + + Class analyzerClass = config.getOptionAsClass(luceneAnalyzerFactoryKey); + + if (analyzerClass != null) { + // get correct constructor + Constructor constructor = analyzerClass.getConstructor(Version.class); + analyzer = (Analyzer)constructor.newInstance(WIKITTY_LUCENE_VERSION); + } + } catch (Exception ex) { + if (log.isErrorEnabled()) { + log.error("Can't init lucene analyzer : " + config.getOption(luceneAnalyzerFactoryKey), ex); + } + + } + + // default to StandardAnalyzer + if (analyzer == null) { + analyzer = new StandardAnalyzer(Version.LUCENE_32); + } + + if (log.isInfoEnabled()) { + log.info("Using lucene analyzer : " + analyzer); + } + + return analyzer; + } + + /* + * @see org.nuiton.wikitty.storage.WikittySearchEngine#clear(org.nuiton.wikitty.services.WikittyTransaction) + */ + @Override + public void clear(WikittyTransaction transaction) throws WikittyException { + + // TODO manage transaction + + IndexWriter writer = null; + try { + writer = new IndexWriter(indexDirectory, indexWriterConfig); + writer.deleteAll(); + writer.commit(); + } catch (IOException ex) { + try { + writer.rollback(); + } catch (IOException e) { + throw new WikittyException("Can't rollback index", ex); + } + throw new WikittyException("Can't clear index", ex); + } finally { + IOUtils.closeQuietly(writer); + } + + } + + /* + * @see org.nuiton.wikitty.storage.WikittySearchEngine#store(org.nuiton.wikitty.services.WikittyTransaction, java.util.Collection, boolean) + */ + @Override + public void store(WikittyTransaction transaction, + Collection<Wikitty> wikitties, boolean force) throws WikittyException { + + // TODO manage transaction + + IndexWriter writer = null; + try { + writer = new IndexWriter(indexDirectory, indexWriterConfig); + + for (Wikitty w : wikitties) { + Document wDoc = convertWikittyToDoc(w); + writer.addDocument(wDoc); + } + + writer.commit(); + + } catch (IOException ex) { + try { + writer.rollback(); + } catch (IOException e) { + throw new WikittyException("Can't rollback index", ex); + } + throw new WikittyException("Can't store to index", ex); + } finally { + IOUtils.closeQuietly(writer); + } + } + + /** + * Convert a wikitty object to a lucene {@link Document}. + * + * @param w wikitty to convert + * @return lucene document + */ + protected Document convertWikittyToDoc(Wikitty w) { + + Document document = new Document(); + + // wikitty id + Field luceneIdField = new Field(WikittyLuceneConstants.LUCENE_ID, w.getId(), + Store.YES, Index.ANALYZED); + document.add(luceneIdField); + + // all other wikitty fields + for (String wikyttyField : w.getAllFieldNames()) { + String ext = WikittyUtil.getExtensionNameFromFQFieldName(wikyttyField); + String fieldName = WikittyUtil.getFieldNameFromFQFieldName(wikyttyField); + String value = w.getFieldAsString(ext, fieldName); + + // Store.YES = mandatory for facets + // Index.ANALYZED = mandatory for search on field + Field luceneField = new Field(wikyttyField, value, Store.YES, Index.ANALYZED); + document.add(luceneField); + } + + return document; + } + + /* + * @see org.nuiton.wikitty.storage.WikittySearchEngine#delete(org.nuiton.wikitty.services.WikittyTransaction, java.util.Collection) + */ + @Override + public void delete(WikittyTransaction transaction, Collection<String> idList) + throws WikittyException { + + // TODO manage transaction + + IndexWriter writer = null; + try { + writer = new IndexWriter(indexDirectory, indexWriterConfig); + + for (String id : idList) { + Term term = new Term(WikittyLuceneConstants.LUCENE_ID, id); + writer.deleteDocuments(term); + } + + writer.commit(); + + } catch (IOException ex) { + try { + writer.rollback(); + } catch (IOException e) { + throw new WikittyException("Can't rollback index", ex); + } + throw new WikittyException("Can't store to index", ex); + } finally { + IOUtils.closeQuietly(writer); + } + } + + /* + * @see org.nuiton.wikitty.storage.WikittySearchEngine#findAllByCriteria(org.nuiton.wikitty.services.WikittyTransaction, org.nuiton.wikitty.search.Criteria) + */ + @Override + public PagedResult<String> findAllByCriteria( + WikittyTransaction transaction, Criteria criteria) throws WikittyException { + + // TODO manage transaction + + PagedResult<String> pagedResult = null; + IndexSearcher searcher = null; + try { + searcher = new IndexSearcher(indexDirectory); + + // get lucene query + Restriction restriction = criteria.getRestriction(); + Restriction2Lucene restriction2Lucene = new Restriction2Lucene(indexAnalyzer); + Query query = restriction2Lucene.toLucene(restriction); + + // configure sorting + List<SortField> sortFields = new ArrayList<SortField>(); + List<String> sortAscending = criteria.getSortAscending(); + if (sortAscending != null) { + for (String sort : sortAscending) { + //String tranform = fieldModifier.convertToSolr(transaction, sort); + //tranform += WikittySolrConstant.SUFFIX_SORTABLE; + SortField sortField = new SortField(sort, SortField.STRING); + sortFields.add(sortField); + } + } + List<String> sortDescending = criteria.getSortDescending(); + if (sortDescending != null) { + for (String sort : sortDescending) { + //String tranform = fieldModifier.convertToSolr(transaction, sort); + //tranform += WikittySolrConstant.SUFFIX_SORTABLE; + SortField sortField = new SortField(sort, SortField.STRING, true); + sortFields.add(sortField); + } + } + Sort sortOption = new Sort(sortAscending.toArray(new SortField[sortAscending.size()])); + + // get search offset and limit + int firstIndex = criteria.getFirstIndex(); + int endIndex = criteria.getEndIndex(); + if (endIndex < 0) { + endIndex = searcher.maxDoc(); + } + + // get results + TopDocs topDocs = searcher.search(query, null, endIndex, sortOption); + ScoreDoc[] scoreDocs = topDocs.scoreDocs; + + int currentDocIndex = 0; + List<String> results = new ArrayList<String>(); + for (ScoreDoc scoreDoc : scoreDocs) { + if (currentDocIndex >= firstIndex && currentDocIndex <= endIndex) { + int luceneId = scoreDoc.doc; + Document document = searcher.doc(luceneId); + String wikittyId = document.get(WikittyLuceneConstants.LUCENE_ID); + results.add(wikittyId); + } + } + + pagedResult = new PagedResult<String>(firstIndex, topDocs.totalHits, query.toString(), null, results); + + } catch (IOException ex) { + throw new WikittyException("Can't search on index", ex); + } finally { + IOUtils.closeQuietly(searcher); + } + + return pagedResult; + } + + /* + * @see org.nuiton.wikitty.storage.WikittySearchEngine#findAllChildrenCount(org.nuiton.wikitty.services.WikittyTransaction, java.lang.String, int, boolean, org.nuiton.wikitty.search.Criteria) + */ + @Override + public TreeNodeResult<String> findAllChildrenCount( + WikittyTransaction transaction, String wikittyId, int depth, + boolean count, Criteria filter) { + + return null; + } + +} Property changes on: trunk/wikitty-lucene/src/main/java/org/nuiton/wikitty/storage/lucene/WikittySearchEngineLucene.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/WikittySearchEngineLuceneTest.java =================================================================== --- trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/WikittySearchEngineLuceneTest.java (rev 0) +++ trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/WikittySearchEngineLuceneTest.java 2011-06-22 09:47:11 UTC (rev 959) @@ -0,0 +1,377 @@ +/* + * #%L + * Wikitty :: wikitty-lucene + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 CodeLutin, Chatellier Eric + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Lesser Public License for more details. + * + * You should have received a copy of the GNU General Lesser Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/lgpl-3.0.html>. + * #L% + */ + +package org.nuiton.wikitty.storage.lucene; + +import java.io.File; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Assert; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; +import org.nuiton.util.ApplicationConfig; +import org.nuiton.wikitty.WikittyConfig; +import org.nuiton.wikitty.WikittyConfigOption; +import org.nuiton.wikitty.WikittyProxy; +import org.nuiton.wikitty.entities.Wikitty; +import org.nuiton.wikitty.entities.WikittyImpl; +import org.nuiton.wikitty.entities.WikittyLabel; +import org.nuiton.wikitty.entities.WikittyLabelImpl; +import org.nuiton.wikitty.entities.WikittyTreeNode; +import org.nuiton.wikitty.entities.WikittyTreeNodeImpl; +import org.nuiton.wikitty.entities.WikittyUserImpl; +import org.nuiton.wikitty.search.Criteria; +import org.nuiton.wikitty.search.PagedResult; +import org.nuiton.wikitty.search.Search; +import org.nuiton.wikitty.search.TreeNodeResult; +import org.nuiton.wikitty.search.operators.Element; +import org.nuiton.wikitty.services.WikittyEvent; + +/** + * Test for class {@link WikittySearchEngineLucene}. + * + * @author poussin + * @version $Revision$ + * + * Last update: $Date$ + * by : $Author$ + */ +public class WikittySearchEngineLuceneTest { + + /** to use log facility, just put in your code: log.info(\"...\"); */ + private static final Log log = LogFactory.getLog(WikittySearchEngineLuceneTest.class); + + protected static WikittyServiceLucene ws; + + protected static ApplicationConfig instance = WikittyConfig.getConfig("wikitty-config-sample-inmemory.properties"); + + protected WikittyProxy proxy = new WikittyProxy(ws); + + @BeforeClass + public static void initTests() throws Exception { + String tmpdir = System.getProperty("java.io.tmpdir"); + File indexDir = new File(tmpdir, "test"); + instance = WikittyConfig.getConfig("wikitty-config-sample-inmemory.properties"); + instance.setOption(WikittyConfigOption.WIKITTY_DATA_DIR.key, indexDir.getAbsolutePath()); + + ws = new WikittyServiceLucene(instance); + } + + @Before + public void deleteAll() throws Exception { + ws.clear(null); + } + + @Test + public void testFullTextSearch() throws Exception { + // for id for easy debugging + WikittyImpl w = new WikittyImpl("label"); + WikittyLabelImpl l = new WikittyLabelImpl(w); + l.addLabels("label"); + w = new WikittyImpl("LABEL"); + WikittyLabelImpl l2 = new WikittyLabelImpl(w); + l2.addLabels("OTHER LABEL"); + w = new WikittyImpl("TATA"); + WikittyUserImpl u = new WikittyUserImpl(w); + u.setLogin("tata"); + + List<Wikitty> toStore = new ArrayList<Wikitty>(); + toStore.add(l.getWikitty()); + toStore.add(l2.getWikitty()); + toStore.add(u.getWikitty()); + + ws.store(null, toStore, false); + + { + Criteria c = Search.query().keyword("lab*").criteria(); + PagedResult<String> result = ws.findAllByCriteria( + null, Collections.singletonList(c)).get(0); + System.out.println(result.getAll()); + Assert.assertEquals(2, result.getNumFound()); + } + { + Criteria c = Search.query().keyword("*a*").criteria(); + PagedResult<String> result = ws.findAllByCriteria( + null, Collections.singletonList(c)).get(0); + System.out.println(result.getAll()); + Assert.assertEquals(3, result.getNumFound()); + } + + } + + @Test + public void testCountAttachment() throws Exception { + List<Wikitty> toStore = new ArrayList<Wikitty>(); + + WikittyImpl w1 = new WikittyImpl("at1"); + WikittyImpl w2 = new WikittyImpl("at2"); + WikittyImpl w3 = new WikittyImpl("at3"); + + WikittyImpl root = new WikittyImpl("the-root"); + WikittyTreeNodeImpl n = new WikittyTreeNodeImpl(root); + n.addAttachment(w1.getId()); + n.addAttachment(w2.getId()); + n.addAttachment(w3.getId()); + + toStore.add(w1); + toStore.add(w2); + toStore.add(w3); + toStore.add(root); + + ws.store(null, toStore, false); + toStore.clear(); + + WikittyImpl w11 = new WikittyImpl("at11"); + WikittyImpl w12 = new WikittyImpl("at12"); + WikittyImpl w13 = new WikittyImpl("at13"); + + WikittyImpl node1 = new WikittyImpl("node1"); + n = new WikittyTreeNodeImpl(node1); + n.setParent(root.getId()); + n.addAttachment(w11.getId()); + n.addAttachment(w12.getId()); + n.addAttachment(w13.getId()); + + toStore.add(w11); + toStore.add(w12); + toStore.add(w13); + toStore.add(node1); + + ws.store(null, toStore, false); + toStore.clear(); + + WikittyImpl w21 = new WikittyImpl("at21"); + WikittyImpl w22 = new WikittyImpl("at22"); + WikittyImpl w23 = new WikittyImpl("at23"); + WikittyImpl w24 = new WikittyImpl("at24"); + + WikittyImpl node2 = new WikittyImpl("node2"); + n = new WikittyTreeNodeImpl(node2); + n.setParent(root.getId()); + n.addAttachment(w21.getId()); + n.addAttachment(w22.getId()); + n.addAttachment(w23.getId()); + n.addAttachment(w24.getId()); + + toStore.add(w21); + toStore.add(w22); + toStore.add(w23); + toStore.add(w24); + toStore.add(node2); + + ws.store(null, toStore, false); + toStore.clear(); + + WikittyImpl w111 = new WikittyImpl("at111"); + WikittyImpl w112 = new WikittyImpl("at112"); + + WikittyImpl node11 = new WikittyImpl("node11"); + n = new WikittyTreeNodeImpl(node11); + n.setParent(node1.getId()); + n.addAttachment(w111.getId()); + n.addAttachment(w112.getId()); + + toStore.add(w111); + toStore.add(w112); + toStore.add(node11); + + ws.store(null, toStore, false); + toStore.clear(); + + TreeNodeResult<String> treeNodeResult = + ws.getSearchEngine().findAllChildrenCount( + null, root.getId(), 0, true, null); + int val = treeNodeResult.getAttCount(); +// Integer val = ws.getSearchEngine().findNodeCount(null, root, null); + Assert.assertEquals(12, val); + +// Map<WikittyTreeNode, Integer> children = ws.findTreeNode(null, root.getId(), null); +// Map<String, Integer> children = ws.getSearchEngine().findAllChildrenCount(null, root, null); + TreeNodeResult<String> children = + ws.getSearchEngine().findAllChildrenCount( + null, root.getId(), 1, true, null); + if (log.isDebugEnabled()) { + log.debug("Children : " + children); + } + + Assert.assertEquals(2, children.getChildCount()); + Assert.assertEquals(5, children.getChild(node1.getId()).getAttCount()); + Assert.assertEquals(4, children.getChild(node2.getId()).getAttCount()); + } + + /** + * Test que lors de la reindexation les noeuds indexés le sont + * correctement si leur attachement ne sont pas encore + * présent dans l'index. + * + * Stocke les attachements avant le noeud. + */ + @Test + public void testReindexWithAttachement() { + + // store attachement + Wikitty attach1 = new WikittyImpl("att1"); + Wikitty attach2 = new WikittyImpl("att2"); + Collection<Wikitty> toStore = new ArrayList<Wikitty>(); + toStore.add(attach1); + toStore.add(attach2); + ws.store(null, toStore, false); + + // store first node + Wikitty treeNode = new WikittyImpl(); + WikittyTreeNode treeNodeImpl = new WikittyTreeNodeImpl(treeNode); + treeNodeImpl.setName("root"); + treeNodeImpl.addAttachment(attach1.getId()); + treeNodeImpl.addAttachment(attach2.getId()); + ws.store(null, Collections.singleton(treeNode), false); + + // set resync + ws.syncSearchEngine(null); + + // search wikitty with attachement + Search query = Search.query(); + query.eq(Element.ELT_EXTENSION, WikittyTreeNodeImpl.EXT_WIKITTYTREENODE); + query.eq(WikittyTreeNode.FQ_FIELD_WIKITTYTREENODE_NAME, "root"); + PagedResult<Wikitty> results = proxy.findAllByCriteria(query.criteria()); + Assert.assertEquals(1, results.size()); + WikittyTreeNode resultTreeNode = new WikittyTreeNodeImpl(results.get(0)); + Assert.assertEquals(2, resultTreeNode.getAttachment().size()); + } + + /** + * Test que lors de la reindexation les noeuds indexés le sont + * correctement si leur attachement ne sont pas encore + * présent dans l'index. + * + * Stocke le noeud avant les attachements. + */ + @Test + public void testReindexWithAttachementOrdering() { + + // store first node + Wikitty treeNode = new WikittyImpl(); + WikittyTreeNode treeNodeImpl = new WikittyTreeNodeImpl(treeNode); + treeNodeImpl.setName("root"); + WikittyEvent event = ws.store(null, Collections.singleton(treeNode), false); + treeNode = event.getWikitties().get(treeNode.getId()); + treeNodeImpl = new WikittyTreeNodeImpl(treeNode); + + // store attachement + Wikitty attach1 = new WikittyImpl("att1"); + Wikitty attach2 = new WikittyImpl("att2"); + treeNodeImpl.addAttachment(attach1.getId()); + treeNodeImpl.addAttachment(attach2.getId()); + Collection<Wikitty> toStore = new ArrayList<Wikitty>(); + toStore.add(attach1); + toStore.add(attach2); + toStore.add(treeNode); + ws.store(null, toStore, false); + + // set resync + ws.syncSearchEngine(null); + + // search wikitty with attachement + Search query = Search.query(); + query.eq(Element.ELT_EXTENSION, WikittyTreeNodeImpl.EXT_WIKITTYTREENODE); + query.eq(WikittyTreeNode.FQ_FIELD_WIKITTYTREENODE_NAME, "root"); + PagedResult<Wikitty> results = proxy.findAllByCriteria(query.criteria()); + Assert.assertEquals(1, results.size()); + WikittyTreeNode resultTreeNode = new WikittyTreeNodeImpl(results.get(0)); + Assert.assertEquals(2, resultTreeNode.getAttachment().size()); + } + + /** + * Test que lorsqu'un document solr est supprimé, il l'est réelement + * car le proxy gere les retours null pour des documents qui n'existe + * plus dans le base, mais les recherches continue de les trouver. + */ + @Test + public void testSolrDeleteDocument() { + // store new wikitty + Wikitty toDeleteWikitty = new WikittyImpl("wikkitytodelete"); + ws.store(null, Collections.singleton(toDeleteWikitty), false); + // look for it + Search query = Search.query(); + query.eq(Element.ELT_ID, "wikkitytodelete"); + Assert.assertEquals(1, ws.findAllByCriteria(null, + Collections.singletonList(query.criteria())).get(0).getNumFound()); + Assert.assertEquals(1, proxy.findAllByCriteria(query.criteria()).getNumFound()); + + // delete it + ws.delete(null, Collections.singleton(toDeleteWikitty.getId())); + + // try to look for it after deletion + Assert.assertEquals(0, proxy.findAllByCriteria(query.criteria()).getNumFound()); + Assert.assertEquals(0, ws.findAllByCriteria(null, + Collections.singletonList(query.criteria())).get(0).getNumFound()); + } + + // Nombre of iteration for test delete + public static final int NB_DOCS_TO_DELETE = 17; + + /** + * Test que lorsque plus de 10 documents solr sont supprimés, il le sont réelement + * car par défault, solr limte les resultats à 10 + */ + @Test + public void testSolrDeleteMoreThan10Documents() { + + List<WikittyLabel> toDelete = new ArrayList<WikittyLabel>(); + + for (int i = 0;i < NB_DOCS_TO_DELETE;i++) { + WikittyLabel toDeleteEntity = new WikittyLabelImpl(); + + // store new wikitty + toDeleteEntity.addLabels("toDeleteEntity" + i); + toDelete.add(toDeleteEntity); + } + + // Store all + proxy.store(toDelete); + + // look for it + Search query = Search.query(); + query.eq(Element.ELT_EXTENSION, WikittyLabel.EXT_WIKITTYLABEL); + Assert.assertEquals(NB_DOCS_TO_DELETE, ws.findAllByCriteria(null, + Collections.singletonList(query.criteria())).get(0).getNumFound()); + Assert.assertEquals(NB_DOCS_TO_DELETE, proxy.findAllByCriteria(query.criteria()).getNumFound()); + Assert.assertEquals(NB_DOCS_TO_DELETE, proxy.findAllByCriteria(WikittyLabel.class, query.criteria()).getNumFound()); + + // delete it + proxy.delete(toDelete); + + // try to look for it after deletion + Assert.assertEquals(0, proxy.findAllByCriteria(query.criteria()).getNumFound()); + Assert.assertEquals(0, ws.findAllByCriteria(null, + Collections.singletonList(query.criteria())).get(0).getNumFound()); + Assert.assertEquals(0, proxy.findAllByCriteria(WikittyLabel.class, query.criteria()).size()); + } +} Property changes on: trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/WikittySearchEngineLuceneTest.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/WikittyServiceLucene.java =================================================================== --- trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/WikittyServiceLucene.java (rev 0) +++ trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/WikittyServiceLucene.java 2011-06-22 09:47:11 UTC (rev 959) @@ -0,0 +1,48 @@ +/* + * #%L + * Wikitty :: wikitty-lucene + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 CodeLutin, Chatellier Eric + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Lesser Public License for more details. + * + * You should have received a copy of the GNU General Lesser Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/lgpl-3.0.html>. + * #L% + */ + +package org.nuiton.wikitty.storage.lucene; + +import org.nuiton.util.ApplicationConfig; +import org.nuiton.wikitty.services.WikittyServiceStorage; +import org.nuiton.wikitty.storage.WikittyExtensionStorageInMemory; +import org.nuiton.wikitty.storage.WikittyStorageInMemory; + +/** + * Wikitty service utilisant lucene comme search engine. + * + * @author chatellier + * @version $Revision$ + * + * Last update: $Date$ + * by : $Author$ + */ +public class WikittyServiceLucene extends WikittyServiceStorage { + public WikittyServiceLucene(ApplicationConfig config) { + extensionStorage = new WikittyExtensionStorageInMemory(); + wikittyStorage = new WikittyStorageInMemory(); + searchEngine = new WikittySearchEngineLucene(config, extensionStorage); + } +} Property changes on: trunk/wikitty-lucene/src/test/java/org/nuiton/wikitty/storage/lucene/WikittyServiceLucene.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL