branch develop updated (43e6198 -> 714ca74)
This is an automated email from the git hooks/post-receive script. New change to branch develop in repository tutti. See http://git.codelutin.com/tutti.git from 43e6198 fixes #6594: [REFERENTIEL] Pb de mise à jour (utilisation adagio 3.8.3.1) new e33b29e use codelutinpom wagon-m-p version new 740d00d introduction de classes différentes pour les différentes types de signes new d321712 deplacement du code csv dabs un paquetage dédié + nettoyage code new 412db54 ajout d'une méthode pour retrouver tous les lots d'un lot racine new f9c70c9 deplacement du code csv dabs un paquetage dédié + nettoyage code new 11bae16 introduction d'un objet context de l'import new f11ac3a suppression des vieux signes new 8e2435c réusinage du code d'import et prise en compte du nouveau mode d'import (en indiquant le lot plutot que l'espèce carégorisée) new 714ca74 fixes #6471 The 9 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Detailed log of new commits: commit 714ca74249a7e13f8968504fbc586789a5dcde8c Merge: e33b29e 8e2435c Author: Tony CHEMIT <chemit@codelutin.com> Date: Wed Feb 4 11:34:39 2015 +0100 fixes #6471 commit e33b29ed09fcdcd1f897d9b3171bd54e260be846 Author: Tony CHEMIT <chemit@codelutin.com> Date: Wed Feb 4 11:34:03 2015 +0100 use codelutinpom wagon-m-p version commit 8e2435c767b55bf3171a1ea9fe98fe8b9272234d Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:15:39 2015 +0100 réusinage du code d'import et prise en compte du nouveau mode d'import (en indiquant le lot plutot que l'espèce carégorisée) commit f11ac3ab84c28f8550457d523c752d2c8a4ec876 Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:13:42 2015 +0100 suppression des vieux signes commit 11bae16b4b4318842ed0cc14c12ac59bb13e47bc Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:13:23 2015 +0100 introduction d'un objet context de l'import commit f9c70c91635c4bac180c29bcf2a609719685ddc2 Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:13:05 2015 +0100 deplacement du code csv dabs un paquetage dédié + nettoyage code commit 412db5483e727fc6a109450da37c82c747d5dbf7 Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:12:42 2015 +0100 ajout d'une méthode pour retrouver tous les lots d'un lot racine commit d321712f02bce8788e06806c501f439de9e5b2a9 Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:12:13 2015 +0100 deplacement du code csv dabs un paquetage dédié + nettoyage code commit 740d00d919faf35b49d0cd8b215e298a59d4d120 Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:11:34 2015 +0100 introduction de classes différentes pour les différentes types de signes Summary of changes: pom.xml | 1 - .../persistence/entities/data/SpeciesBatchs.java | 20 ++ .../tutti/service/bigfin/BigfinDataRowModel.java | 138 -------- .../tutti/service/bigfin/BigfinImportContext.java | 357 ++++++++++++++++++++ .../tutti/service/bigfin/BigfinImportService.java | 366 +++++++++------------ .../fr/ifremer/tutti/service/bigfin/Signs.java | 216 ------------ .../service/bigfin/{ => csv}/BigfinDataRow.java | 75 ++--- .../service/bigfin/csv/BigfinDataRowModel.java | 231 +++++++++++++ .../service/bigfin/csv/SpeciesOrSpeciesBatch.java | 42 +++ .../fr/ifremer/tutti/service/bigfin/signs/Sex.java | 109 ++++++ .../ifremer/tutti/service/bigfin/signs/Sign.java | 27 ++ .../ifremer/tutti/service/bigfin/signs/Size.java | 111 +++++++ .../tutti/service/bigfin/signs/VracHorsVrac.java | 92 ++++++ .../resources/i18n/tutti-service_en_GB.properties | 1 + .../resources/i18n/tutti-service_fr_FR.properties | 3 +- 15 files changed, 1186 insertions(+), 603 deletions(-) delete mode 100644 tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinDataRowModel.java create mode 100644 tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinImportContext.java delete mode 100644 tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/Signs.java rename tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/{ => csv}/BigfinDataRow.java (60%) create mode 100644 tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/BigfinDataRowModel.java create mode 100644 tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/SpeciesOrSpeciesBatch.java create mode 100644 tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Sex.java create mode 100644 tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Sign.java create mode 100644 tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Size.java create mode 100644 tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/VracHorsVrac.java -- To stop receiving notification emails like this one, please contact codelutin.com SCM administrator <admin+scm@codelutin.com>.
This is an automated email from the git hooks/post-receive script. New commit to branch develop in repository tutti. See http://git.codelutin.com/tutti.git commit 740d00d919faf35b49d0cd8b215e298a59d4d120 Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:11:34 2015 +0100 introduction de classes différentes pour les différentes types de signes --- .../fr/ifremer/tutti/service/bigfin/signs/Sex.java | 109 ++++++++++++++++++++ .../ifremer/tutti/service/bigfin/signs/Sign.java | 27 +++++ .../ifremer/tutti/service/bigfin/signs/Size.java | 111 +++++++++++++++++++++ .../tutti/service/bigfin/signs/VracHorsVrac.java | 92 +++++++++++++++++ 4 files changed, 339 insertions(+) diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Sex.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Sex.java new file mode 100644 index 0000000..cce79c8 --- /dev/null +++ b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Sex.java @@ -0,0 +1,109 @@ +package fr.ifremer.tutti.service.bigfin.signs; + +import com.google.common.base.Function; +import fr.ifremer.adagio.core.dao.referential.pmfm.PmfmId; +import fr.ifremer.adagio.core.dao.referential.pmfm.QualitativeValueId; +import fr.ifremer.tutti.persistence.entities.referential.Caracteristic; +import fr.ifremer.tutti.persistence.entities.referential.CaracteristicQualitativeValue; +import fr.ifremer.tutti.persistence.entities.referential.CaracteristicQualitativeValues; +import fr.ifremer.tutti.service.bigfin.csv.BigfinDataRow; + +import java.util.Map; + +/** + * Created on 2/3/15. + * + * @author Tony Chemit - chemit@codelutin.com + * @since 3.13 + */ +public enum Sex implements Sign { + + NONE("NONE") { + @Override + public Integer getCategory() { + return PmfmId.SEX.getValue(); + } + + @Override + public Integer getQualitativeValueId() { + return QualitativeValueId.NON_SEXED_SEX.getValue(); + } + + @Override + public boolean isNullEquivalent() { + return false; + } + }, + MALE("MALE") { + @Override + public Integer getCategory() { + return PmfmId.SEX.getValue(); + } + + @Override + public Integer getQualitativeValueId() { + return QualitativeValueId.SEX_MALE.getValue(); + } + + @Override + public boolean isNullEquivalent() { + return false; + } + }, + FEMALE("FEMALE") { + @Override + public Integer getCategory() { + return PmfmId.SEX.getValue(); + } + + @Override + public Integer getQualitativeValueId() { + return QualitativeValueId.SEX_FEMALE.getValue(); + } + + @Override + public boolean isNullEquivalent() { + return false; + } + }; + + private String sign; + + Sex(String sign) { + this.sign = sign; + } + + @Override + public String getSign() { + return sign; + } + + @Override + public void registerSign(Caracteristic caracteristic, Map<Sign, CaracteristicQualitativeValue> map) { + Integer valueId = getQualitativeValueId(); + CaracteristicQualitativeValue result = CaracteristicQualitativeValues.getQualitativeValue(caracteristic, valueId); + map.put(this, result); + } + + public static Sex getValue(String sign) { + Sex result = null; + for (Sex s : values()) { + if (s.sign.equals(sign)) { + result = s; + break; + } + } + return result; + } + + public static Function<BigfinDataRow, Sign> newExtractValueFunction() { + return new Function<BigfinDataRow, Sign>() { + @Override + public Sign apply(BigfinDataRow bigfinDataRow) { + Sign result = bigfinDataRow.getSex(); + return result; + } + }; + } + +} diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Sign.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Sign.java new file mode 100644 index 0000000..d735381 --- /dev/null +++ b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Sign.java @@ -0,0 +1,27 @@ +package fr.ifremer.tutti.service.bigfin.signs; + +import fr.ifremer.tutti.persistence.entities.referential.Caracteristic; +import fr.ifremer.tutti.persistence.entities.referential.CaracteristicQualitativeValue; + +import java.util.Map; + +/** + * Created on 2/3/15. + * + * @author Tony Chemit - chemit@codelutin.com + * @since 3.13 + */ +public interface Sign { + + public abstract Integer getCategory(); + + public abstract Integer getQualitativeValueId(); + + // if true, can use this value in the import file to replace a skipped category + public abstract boolean isNullEquivalent(); + + public String getSign(); + + public void registerSign(Caracteristic caracteristic, Map<Sign, CaracteristicQualitativeValue> map); + +} diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Size.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Size.java new file mode 100644 index 0000000..065f20c --- /dev/null +++ b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/Size.java @@ -0,0 +1,111 @@ +package fr.ifremer.tutti.service.bigfin.signs; + +import com.google.common.base.Function; +import fr.ifremer.adagio.core.dao.referential.pmfm.PmfmId; +import fr.ifremer.adagio.core.dao.referential.pmfm.QualitativeValueId; +import fr.ifremer.tutti.persistence.entities.referential.Caracteristic; +import fr.ifremer.tutti.persistence.entities.referential.CaracteristicQualitativeValue; +import fr.ifremer.tutti.persistence.entities.referential.CaracteristicQualitativeValues; +import fr.ifremer.tutti.service.bigfin.csv.BigfinDataRow; + +import java.util.Map; + +/** + * Created on 2/3/15. + * + * @author Tony Chemit - chemit@codelutin.com + * @since 3.13 + */ +public enum Size implements Sign { + + // classe de taille, 1 = petit ; 2 = gros ; 0 = pas de classe de taille (saisie libre donc risque fort de mauvaise saisie) + NOT_SIZED("0") { + @Override + public Integer getCategory() { + return PmfmId.SIZE_CATEGORY.getValue(); + } + + @Override + public Integer getQualitativeValueId() { + return QualitativeValueId.UNSORTED.getValue(); + } + + @Override + public boolean isNullEquivalent() { + return true; + } + + }, + SMALL("1") { + @Override + public Integer getCategory() { + return PmfmId.SIZE_CATEGORY.getValue(); + } + + @Override + public Integer getQualitativeValueId() { + return QualitativeValueId.SIZE_SMALL.getValue(); + } + + @Override + public boolean isNullEquivalent() { + return false; + } + }, + BIG("2") { + @Override + public Integer getCategory() { + return PmfmId.SIZE_CATEGORY.getValue(); + } + + @Override + public Integer getQualitativeValueId() { + return QualitativeValueId.SIZE_BIG.getValue(); + } + + @Override + public boolean isNullEquivalent() { + return false; + } + }; + + private String sign; + + Size(String sign) { + this.sign = sign; + } + + @Override + public String getSign() { + return sign; + } + + @Override + public void registerSign(Caracteristic caracteristic, Map<Sign, CaracteristicQualitativeValue> map) { + Integer valueId = getQualitativeValueId(); + CaracteristicQualitativeValue result = CaracteristicQualitativeValues.getQualitativeValue(caracteristic, valueId); + map.put(this, result); + } + + public static Size getValue(String sign) { + Size result = null; + for (Size s : values()) { + if (s.sign.equals(sign)) { + result = s; + break; + } + } + return result; + } + + public static Function<BigfinDataRow, Sign> newExtractValueFunction() { + return new Function<BigfinDataRow, Sign>() { + @Override + public Sign apply(BigfinDataRow bigfinDataRow) { + Sign result = bigfinDataRow.getSize(); + return result; + } + }; + } + +} diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/VracHorsVrac.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/VracHorsVrac.java new file mode 100644 index 0000000..27b9cbe --- /dev/null +++ b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/signs/VracHorsVrac.java @@ -0,0 +1,92 @@ +package fr.ifremer.tutti.service.bigfin.signs; + +import com.google.common.base.Function; +import fr.ifremer.adagio.core.dao.referential.pmfm.PmfmId; +import fr.ifremer.adagio.core.dao.referential.pmfm.QualitativeValueId; +import fr.ifremer.tutti.persistence.entities.referential.Caracteristic; +import fr.ifremer.tutti.persistence.entities.referential.CaracteristicQualitativeValue; +import fr.ifremer.tutti.persistence.entities.referential.CaracteristicQualitativeValues; +import fr.ifremer.tutti.service.bigfin.csv.BigfinDataRow; + +import java.util.Map; + +/** + * Created on 2/3/15. + * + * @author Tony Chemit - chemit@codelutin.com + * @since 3.13 + */ +public enum VracHorsVrac implements Sign { + + VRAC("") { + @Override + public Integer getCategory() { + return PmfmId.SORTED_UNSORTED.getValue(); + } + + @Override + public Integer getQualitativeValueId() { + return QualitativeValueId.SORTED_VRAC.getValue(); + } + + @Override + public boolean isNullEquivalent() { + return false; + } + }, + HORS_VRAC("HV") { + @Override + public Integer getCategory() { + return PmfmId.SORTED_UNSORTED.getValue(); + } + + @Override + public Integer getQualitativeValueId() { + return QualitativeValueId.SORTED_HORS_VRAC.getValue(); + } + + @Override + public boolean isNullEquivalent() { + return false; + } + }; + + private String sign; + + VracHorsVrac(String sign) { + this.sign = sign; + } + + @Override + public String getSign() { + return sign; + } + + @Override + public void registerSign(Caracteristic caracteristic, Map<Sign, CaracteristicQualitativeValue> map) { + Integer valueId = getQualitativeValueId(); + CaracteristicQualitativeValue result = CaracteristicQualitativeValues.getQualitativeValue(caracteristic, valueId); + map.put(this, result); + } + + public static VracHorsVrac getValue(String sign) { + VracHorsVrac result = null; + for (VracHorsVrac s : values()) { + if (s.sign.equals(sign)) { + result = s; + break; + } + } + return result; + } + + public static Function<BigfinDataRow, Sign> newExtractValueFunction() { + return new Function<BigfinDataRow, Sign>() { + @Override + public Sign apply(BigfinDataRow bigfinDataRow) { + Sign result = bigfinDataRow.getVracHorsVrac(); + return result; + } + }; + } +} -- To stop receiving notification emails like this one, please contact codelutin.com SCM administrator <admin+scm@codelutin.com>.
This is an automated email from the git hooks/post-receive script. New commit to branch develop in repository tutti. See http://git.codelutin.com/tutti.git commit d321712f02bce8788e06806c501f439de9e5b2a9 Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:12:13 2015 +0100 deplacement du code csv dabs un paquetage dédié + nettoyage code --- .../tutti/service/bigfin/csv/BigfinDataRow.java | 146 +++++++++++++ .../service/bigfin/csv/BigfinDataRowModel.java | 231 +++++++++++++++++++++ .../service/bigfin/csv/SpeciesOrSpeciesBatch.java | 42 ++++ 3 files changed, 419 insertions(+) diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/BigfinDataRow.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/BigfinDataRow.java new file mode 100644 index 0000000..ab7c15b --- /dev/null +++ b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/BigfinDataRow.java @@ -0,0 +1,146 @@ +package fr.ifremer.tutti.service.bigfin.csv; + +/* + * #%L + * Tutti :: Service + * $Id:$ + * $HeadURL:$ + * %% + * Copyright (C) 2012 - 2014 Ifremer + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/gpl-3.0.html>. + * #L% + */ + +import fr.ifremer.tutti.service.bigfin.signs.Sex; +import fr.ifremer.tutti.service.bigfin.signs.Size; +import fr.ifremer.tutti.service.bigfin.signs.VracHorsVrac; + +import java.util.Date; + +/** + * @author Kevin Morin (Code Lutin) + * @since 3.8 + */ +public class BigfinDataRow { + + public static final String PROPERTY_RECORD_ID = "recordId"; + + public static final String PROPERTY_DATE = "date"; + + public static final String PROPERTY_STATION = "station"; + + public static final String PROPERTY_SPECIES_OR_SPECIES_BATCH = "speciesOrSpeciesBatch"; + + public static final String PROPERTY_LENGTH = "length"; + + public static final String PROPERTY_WEIGHT = "weight"; + + public static final String PROPERTY_SIZE = "size"; + + public static final String PROPERTY_SEX = "sex"; + + public static final String PROPERTY_VRAC_HORS_VRAC = "vracHorsVrac"; + + protected String recordId; + + protected Date date; + + protected String station; + + protected SpeciesOrSpeciesBatch speciesOrSpeciesBatch; + + protected float length; + + protected Float weight; + + protected Size size; + + protected Sex sex; + + protected VracHorsVrac vracHorsVrac; + + public String getRecordId() { + return recordId; + } + + public void setRecordId(String recordId) { + this.recordId = recordId; + } + + public float getLength() { + return length; + } + + public void setLength(float length) { + this.length = length; + } + + public Float getWeight() { + return weight; + } + + public void setWeight(Float weight) { + this.weight = weight; + } + + public SpeciesOrSpeciesBatch getSpeciesOrSpeciesBatch() { + return speciesOrSpeciesBatch; + } + + public void setSpeciesOrSpeciesBatch(SpeciesOrSpeciesBatch speciesOrSpeciesBatch) { + this.speciesOrSpeciesBatch = speciesOrSpeciesBatch; + } + + public String getStation() { + return station; + } + + public void setStation(String station) { + this.station = station; + } + + public Date getDate() { + return date; + } + + public void setDate(Date date) { + this.date = date; + } + + public Size getSize() { + return size; + } + + public void setSize(Size size) { + this.size = size; + } + + public Sex getSex() { + return sex; + } + + public void setSex(Sex sex) { + this.sex = sex; + } + + public VracHorsVrac getVracHorsVrac() { + return vracHorsVrac; + } + + public void setVracHorsVrac(VracHorsVrac vracHorsVrac) { + this.vracHorsVrac = vracHorsVrac; + } +} diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/BigfinDataRowModel.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/BigfinDataRowModel.java new file mode 100644 index 0000000..6c378ef --- /dev/null +++ b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/BigfinDataRowModel.java @@ -0,0 +1,231 @@ +package fr.ifremer.tutti.service.bigfin.csv; + +/* + * #%L + * Tutti :: Service + * $Id:$ + * $HeadURL:$ + * %% + * Copyright (C) 2012 - 2014 Ifremer + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/gpl-3.0.html>. + * #L% + */ + +import fr.ifremer.tutti.persistence.entities.data.SpeciesBatch; +import fr.ifremer.tutti.persistence.entities.referential.Species; +import fr.ifremer.tutti.persistence.entities.referential.Speciess; +import fr.ifremer.tutti.service.TuttiCsvUtil; +import fr.ifremer.tutti.service.bigfin.signs.Sex; +import fr.ifremer.tutti.service.bigfin.signs.Size; +import fr.ifremer.tutti.service.bigfin.signs.VracHorsVrac; +import org.apache.commons.lang3.StringUtils; +import org.nuiton.csv.Common; +import org.nuiton.csv.ValueParser; + +import java.text.ParseException; +import java.util.HashMap; +import java.util.Map; + +/** + * @author Kevin Morin (Code Lutin) + * @since 3.8 + */ +public class BigfinDataRowModel extends TuttiCsvUtil.AbstractTuttiImportModel<BigfinDataRow> { + + public BigfinDataRowModel(Map<String, Species> speciesBySurveyCode, Map<Integer, SpeciesBatch> speciesBatchesById) { + + super(','); + + newIgnoredColumn("Study name"); + + newMandatoryColumn("ID", BigfinDataRow.PROPERTY_RECORD_ID); + + // date et heure de l'enregistrement (non importé mais utile pour contrôle à l'import) + newMandatoryColumn("Date", BigfinDataRow.PROPERTY_DATE, new Common.DateValue("MM/dd/yy HH:mm:ss")); + + newIgnoredColumn("LOC"); + + // n° de la station (non importé mais utile pour contrôle à l'import) + newMandatoryColumn("STA", BigfinDataRow.PROPERTY_STATION); + + newMandatoryColumn("COMMENT", BigfinDataRow.PROPERTY_VRAC_HORS_VRAC, new VracHorsVracValueParser()); + + newIgnoredColumn("GPS X"); + + newIgnoredColumn("GPS Y"); + + // code espèce = code campagne (saisie libre donc risque fort de mauvaise saisie) + newMandatoryColumn("SPEC", BigfinDataRow.PROPERTY_SPECIES_OR_SPECIES_BATCH, new SpeciesOrSpeciesBatchValueParser(speciesBySurveyCode, speciesBatchesById)); + + newMandatoryColumn("LENGTH (mm)", BigfinDataRow.PROPERTY_LENGTH, Common.PRIMITIVE_FLOAT); + + newMandatoryColumn("WEIGHT (g)", BigfinDataRow.PROPERTY_WEIGHT, Common.FLOAT); + + newMandatoryColumn("SEX", BigfinDataRow.PROPERTY_SEX, new SexValueParser()); + + // sz class : si code différents de 0 1 ou 2 alors tout bloquer et donner l'id des lignes en anomalies + newMandatoryColumn("SIZE", BigfinDataRow.PROPERTY_SIZE, new SizeValueParser()); + + newIgnoredColumn("MT"); + newIgnoredColumn("MS"); + + } + + @Override + public BigfinDataRow newEmptyInstance() { + return new BigfinDataRow(); + } + + private static class SpeciesOrSpeciesBatchValueParser implements ValueParser<SpeciesOrSpeciesBatch> { + + final Map<String, Species> foundSpecies = new HashMap<>(); + + private final Map<String, Species> speciesBySurveyCode; + + private final Map<Integer, SpeciesBatch> speciesBatches; + + public SpeciesOrSpeciesBatchValueParser(Map<String, Species> speciesBySurveyCode, Map<Integer, SpeciesBatch> speciesBatchesById) { + this.speciesBySurveyCode = speciesBySurveyCode; + this.speciesBatches = speciesBatchesById; + } + + @Override + public SpeciesOrSpeciesBatch parse(String value) throws ParseException { + + SpeciesBatch speciesBatch = null; + Species species = null; + + if (StringUtils.isNotBlank(value)) { + + speciesBatch = getSpeciesBatch(value); + + if (speciesBatch == null) { + + // try a species + species = getSpecies(value); + } + + } + + SpeciesOrSpeciesBatch result; + + if (speciesBatch == null) { + + // Just a species + result = new SpeciesOrSpeciesBatch(species); + } else { + + // SpeciesBatch + result = new SpeciesOrSpeciesBatch(speciesBatch); + } + + return result; + + } + + protected SpeciesBatch getSpeciesBatch(String value) { + + SpeciesBatch speciesBatch = null; + + if (StringUtils.isNotBlank(value)) { + try { + Integer batchId = Integer.valueOf(value); + speciesBatch = speciesBatches.get(batchId); + } catch (NumberFormatException e) { + // Not a number + // Not a species batch + } + + } + + return speciesBatch; + + } + + protected Species getSpecies(String value) { + + Species species; + + if (StringUtils.isBlank(value)) { + + species = Speciess.newSpecies(); + + } else { + + // use upper case + value = value.toUpperCase(); + + // if code already found + species = foundSpecies.get(value); + + //if not found, look for it in the survey codes + if (species == null) { + species = speciesBySurveyCode.get(value); + } + + // Si on ne trouve pas une espèce de code campagne XXXXXXX, on essaye alors avec le code XXXX-XXX + if (species == null) { + String alternativeSpeciesCode = value.substring(0, 4) + '-' + value.substring(4); + species = speciesBySurveyCode.get(alternativeSpeciesCode); + } + + if (species == null) { + species = Speciess.newSpecies(); + species.setExternalCode(value); + } + + // record the code in the found codes + foundSpecies.put(value, species); + + } + + return species; + + } + + } + + private static class SizeValueParser implements ValueParser<Size> { + @Override + public Size parse(String value) throws ParseException { + Size result = Size.getValue(value.toUpperCase()); + return result; + } + } + + private static class SexValueParser implements ValueParser<Sex> { + @Override + public Sex parse(String value) throws ParseException { + Sex result = Sex.getValue(value.toUpperCase()); + if (result == null) { + throw new ParseException("Could not parse Sex value: " + value, 0); + } + return result; + } + } + + private static class VracHorsVracValueParser implements ValueParser<VracHorsVrac> { + @Override + public VracHorsVrac parse(String value) throws ParseException { + // On importe dans le Vrac par défaut, sauf si il y a le texte HV ou hv dans le champ text + VracHorsVrac result = VracHorsVrac.getValue(value.toUpperCase()); + if (result == null) { + result = VracHorsVrac.VRAC; + } + return result; + } + } +} diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/SpeciesOrSpeciesBatch.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/SpeciesOrSpeciesBatch.java new file mode 100644 index 0000000..c7681b1 --- /dev/null +++ b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/csv/SpeciesOrSpeciesBatch.java @@ -0,0 +1,42 @@ +package fr.ifremer.tutti.service.bigfin.csv; + +import fr.ifremer.tutti.persistence.entities.data.SpeciesBatch; +import fr.ifremer.tutti.persistence.entities.referential.Species; + +/** + * Created on 2/3/15. + * + * @author Tony Chemit - chemit@codelutin.com + * @since 3.13 + */ +public class SpeciesOrSpeciesBatch { + + protected Species species; + + protected SpeciesBatch batch; + + public SpeciesOrSpeciesBatch(Species species) { + this.species = species; + } + + public SpeciesOrSpeciesBatch(SpeciesBatch batch) { + this.species = batch.getSpecies(); + this.batch = batch; + } + + public boolean isSpecies() { + return batch == null; + } + + public boolean isBatch() { + return batch != null; + } + + public Species getSpecies() { + return species; + } + + public SpeciesBatch getBatch() { + return batch; + } +} -- To stop receiving notification emails like this one, please contact codelutin.com SCM administrator <admin+scm@codelutin.com>.
This is an automated email from the git hooks/post-receive script. New commit to branch develop in repository tutti. See http://git.codelutin.com/tutti.git commit 412db5483e727fc6a109450da37c82c747d5dbf7 Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:12:42 2015 +0100 ajout d'une méthode pour retrouver tous les lots d'un lot racine --- .../persistence/entities/data/SpeciesBatchs.java | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/tutti-persistence/src/main/java/fr/ifremer/tutti/persistence/entities/data/SpeciesBatchs.java b/tutti-persistence/src/main/java/fr/ifremer/tutti/persistence/entities/data/SpeciesBatchs.java index 3076167..bf7b3d6 100644 --- a/tutti-persistence/src/main/java/fr/ifremer/tutti/persistence/entities/data/SpeciesBatchs.java +++ b/tutti-persistence/src/main/java/fr/ifremer/tutti/persistence/entities/data/SpeciesBatchs.java @@ -27,6 +27,8 @@ package fr.ifremer.tutti.persistence.entities.data; import com.google.common.base.Function; import java.io.Serializable; +import java.util.Map; +import java.util.TreeMap; /** * @author Kevin Morin (Code Lutin) @@ -41,4 +43,22 @@ public class SpeciesBatchs extends AbstractSpeciesBatchs { } }; + public static Map<Integer, SpeciesBatch> getAllSpeciesBatchesById(BatchContainer<SpeciesBatch> rootSpeciesBatch) { + Map<Integer, SpeciesBatch> result = new TreeMap<>(); + + for (SpeciesBatch speciesBatch : rootSpeciesBatch.getChildren()) { + getAllspeciesBatchesById(speciesBatch, result); + } + return result; + } + + private static void getAllspeciesBatchesById(SpeciesBatch speciesBatch, Map<Integer, SpeciesBatch> result) { + result.put(speciesBatch.getIdAsInt(), speciesBatch); + if (!speciesBatch.isChildBatchsEmpty()) { + for (SpeciesBatch batch : speciesBatch.getChildBatchs()) { + getAllspeciesBatchesById(batch, result); + } + } + } + } -- To stop receiving notification emails like this one, please contact codelutin.com SCM administrator <admin+scm@codelutin.com>.
This is an automated email from the git hooks/post-receive script. New commit to branch develop in repository tutti. See http://git.codelutin.com/tutti.git commit f9c70c91635c4bac180c29bcf2a609719685ddc2 Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:13:05 2015 +0100 deplacement du code csv dabs un paquetage dédié + nettoyage code --- .../tutti/service/bigfin/BigfinDataRow.java | 145 --------------------- .../tutti/service/bigfin/BigfinDataRowModel.java | 138 -------------------- 2 files changed, 283 deletions(-) diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinDataRow.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinDataRow.java deleted file mode 100644 index a6cfed0..0000000 --- a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinDataRow.java +++ /dev/null @@ -1,145 +0,0 @@ -package fr.ifremer.tutti.service.bigfin; - -/* - * #%L - * Tutti :: Service - * $Id:$ - * $HeadURL:$ - * %% - * Copyright (C) 2012 - 2014 Ifremer - * %% - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as - * published by the Free Software Foundation, either version 3 of the - * License, or (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public - * License along with this program. If not, see - * <http://www.gnu.org/licenses/gpl-3.0.html>. - * #L% - */ - -import fr.ifremer.tutti.persistence.entities.referential.Species; - -import java.io.Serializable; -import java.util.Date; - -/** - * @author Kevin Morin (Code Lutin) - * @since 3.8 - */ -public class BigfinDataRow implements Serializable { - - public static final String PROPERTY_RECORD_ID = "recordId"; - - public static final String PROPERTY_LENGTH = "length"; - - public static final String PROPERTY_WEIGHT = "weight"; - - public static final String PROPERTY_SPECIES = "species"; - - public static final String PROPERTY_STATION = "station"; - - public static final String PROPERTY_DT = "dt"; - - public static final String PROPERTY_SZ_CLASS = "szClass"; - - public static final String PROPERTY_GENDER = "gender"; - - public static final String PROPERTY_VRAC_HORS_VRAC = "vracHorsVrac"; - - protected String recordId; - - protected float length; - - protected Float weight; - - protected Species species; - - protected String station; - - protected Date dt; - - protected Signs szClass; - - protected Signs gender; - - protected Signs vracHorsVrac; - - public String getRecordId() { - return recordId; - } - - public void setRecordId(String recordId) { - this.recordId = recordId; - } - - public float getLength() { - return length; - } - - public void setLength(float length) { - this.length = length; - } - - public Float getWeight() { - return weight; - } - - public void setWeight(Float weight) { - this.weight = weight; - } - - public Species getSpecies() { - return species; - } - - public void setSpecies(Species species) { - this.species = species; - } - - public String getStation() { - return station; - } - - public void setStation(String station) { - this.station = station; - } - - public Date getDt() { - return dt; - } - - public void setDt(Date dt) { - this.dt = dt; - } - - public Signs getSzClass() { - return szClass; - } - - public void setSzClass(Signs szClass) { - this.szClass = szClass; - } - - public Signs getGender() { - return gender; - } - - public void setGender(Signs gender) { - this.gender = gender; - } - - public Signs getVracHorsVrac() { - return vracHorsVrac; - } - - public void setVracHorsVrac(Signs vracHorsVrac) { - this.vracHorsVrac = vracHorsVrac; - } -} diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinDataRowModel.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinDataRowModel.java deleted file mode 100644 index 4f2b8c1..0000000 --- a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinDataRowModel.java +++ /dev/null @@ -1,138 +0,0 @@ -package fr.ifremer.tutti.service.bigfin; - -/* - * #%L - * Tutti :: Service - * $Id:$ - * $HeadURL:$ - * %% - * Copyright (C) 2012 - 2014 Ifremer - * %% - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as - * published by the Free Software Foundation, either version 3 of the - * License, or (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public - * License along with this program. If not, see - * <http://www.gnu.org/licenses/gpl-3.0.html>. - * #L% - */ - -import fr.ifremer.adagio.core.dao.referential.pmfm.PmfmId; -import fr.ifremer.tutti.persistence.entities.referential.Species; -import fr.ifremer.tutti.persistence.entities.referential.Speciess; -import fr.ifremer.tutti.service.TuttiCsvUtil; -import org.apache.commons.lang3.StringUtils; -import org.nuiton.csv.Common; -import org.nuiton.csv.ValueParser; - -import java.text.ParseException; -import java.util.HashMap; -import java.util.Map; - -/** - * @author Kevin Morin (Code Lutin) - * @since 3.8 - */ -public class BigfinDataRowModel extends TuttiCsvUtil.AbstractTuttiImportModel<BigfinDataRow> { - - public BigfinDataRowModel(char separator, - final Map<String, Species> speciesBySurveyCode) { - - super(separator); - - final Map<String, Species> foundSpecies = new HashMap<>(); - - newMandatoryColumn("Record_id", BigfinDataRow.PROPERTY_RECORD_ID); - newMandatoryColumn("length(mm)", BigfinDataRow.PROPERTY_LENGTH, Common.PRIMITIVE_FLOAT); - newMandatoryColumn("weight(g)", BigfinDataRow.PROPERTY_WEIGHT, Common.FLOAT); - -// code espèce = code campagne (saisie libre donc risque fort de mauvaise saisie) - newMandatoryColumn("species", BigfinDataRow.PROPERTY_SPECIES, new ValueParser<Object>() { - @Override - public Object parse(String value) throws ParseException { - Species result = null; - if (StringUtils.isNotBlank(value)) { - // if code already found - result = foundSpecies.get(value); - - //if not found, look for it in the survey codes - if (result == null) { - result = speciesBySurveyCode.get(value); - } - -// Si on ne trouve pas une espèce de code campagne XXXXXXX, on essaye alors avec le code XXXX-XXX - if (result == null) { - String alternativeSpeciesCode = value.substring(0, 4) + '-' + value.substring(4); - result = speciesBySurveyCode.get(alternativeSpeciesCode); - } - // record the code in the found codes - foundSpecies.put(value, result); - } - // if the species is not found, create a new one with the value in external code - if (result == null) { - result = Speciess.newSpecies(); - result.setExternalCode(value); - } - return result; - } - }); - -// n° de la station (non importé mais utile pour contrôle à l'import) - newMandatoryColumn("station", BigfinDataRow.PROPERTY_STATION); -// date et heure de l'enregistrement (non importé mais utile pour contrôle à l'import) - newMandatoryColumn("Dt", BigfinDataRow.PROPERTY_DT, new Common.DateValue("MM/dd/yy HH:mm")); - -// sz class : si code différents de 0 1 ou 2 alors tout bloquer et donner l'id des lignes en anomalies - newMandatoryColumn("sz class", - BigfinDataRow.PROPERTY_SZ_CLASS, - new ValueParser<Signs>() { - @Override - public Signs parse(String value) throws ParseException { - Signs result = Signs.getSign(value.toUpperCase(), PmfmId.SIZE_CATEGORY.getValue()); - return result; - } - }); - - newMandatoryColumn("gender", - BigfinDataRow.PROPERTY_GENDER, - new ValueParser<Signs>() { - @Override - public Signs parse(String value) throws ParseException { - Signs result = Signs.getSign(value.toUpperCase(), PmfmId.SEX.getValue()); - if (result == null) { - throw new ParseException("Could not parse Sign value: " + value, 0); - } - return result; - } - }); - - newMandatoryColumn("text", BigfinDataRow.PROPERTY_VRAC_HORS_VRAC, new ValueParser<Signs>() { - @Override - public Signs parse(String value) throws ParseException { -// On importe dans le Vrac par défaut, sauf si il y a le texte HV ou hv dans le champ text - Signs result = Signs.getSign(value.toUpperCase(), PmfmId.SORTED_UNSORTED.getValue()); - if (result == null) { - result = Signs.VRAC; - } - return result; - } - }); - - - newIgnoredColumn("cruise"); - newIgnoredColumn("pan"); - } - - @Override - public BigfinDataRow newEmptyInstance() { - return new BigfinDataRow(); - } - -} -- To stop receiving notification emails like this one, please contact codelutin.com SCM administrator <admin+scm@codelutin.com>.
This is an automated email from the git hooks/post-receive script. New commit to branch develop in repository tutti. See http://git.codelutin.com/tutti.git commit 11bae16b4b4318842ed0cc14c12ac59bb13e47bc Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:13:23 2015 +0100 introduction d'un objet context de l'import --- .../tutti/service/bigfin/BigfinImportContext.java | 357 +++++++++++++++++++++ 1 file changed, 357 insertions(+) diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinImportContext.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinImportContext.java new file mode 100644 index 0000000..e295f10 --- /dev/null +++ b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinImportContext.java @@ -0,0 +1,357 @@ +package fr.ifremer.tutti.service.bigfin; + +import com.google.common.base.Function; +import com.google.common.base.Preconditions; +import com.google.common.collect.Multimap; +import com.google.common.collect.Multimaps; +import fr.ifremer.tutti.persistence.entities.data.BatchContainer; +import fr.ifremer.tutti.persistence.entities.data.CatchBatch; +import fr.ifremer.tutti.persistence.entities.data.FishingOperation; +import fr.ifremer.tutti.persistence.entities.data.SpeciesBatch; +import fr.ifremer.tutti.persistence.entities.data.SpeciesBatchs; +import fr.ifremer.tutti.persistence.entities.protocol.SpeciesProtocol; +import fr.ifremer.tutti.persistence.entities.referential.Caracteristic; +import fr.ifremer.tutti.persistence.entities.referential.CaracteristicQualitativeValue; +import fr.ifremer.tutti.persistence.entities.referential.Species; +import fr.ifremer.tutti.service.PersistenceService; +import fr.ifremer.tutti.service.bigfin.csv.BigfinDataRow; +import fr.ifremer.tutti.service.bigfin.csv.SpeciesOrSpeciesBatch; +import fr.ifremer.tutti.service.bigfin.signs.Sign; +import org.apache.commons.lang3.StringUtils; +import org.apache.commons.lang3.time.DateUtils; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import java.io.File; +import java.util.ArrayList; +import java.util.Date; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import static org.nuiton.i18n.I18n.t; + +/** + * Created on 2/3/15. + * + * @author Tony Chemit - chemit@codelutin.com + * @since 3.13 + */ +public class BigfinImportContext { + + /** Logger. */ + private static final Log log = LogFactory.getLog(BigfinImportContext.class); + + protected final FishingOperation operation; + + protected final CatchBatch catchBatch; + + protected final Map<Sign, CaracteristicQualitativeValue> signsToCaracteristicValue; + + protected final Map<String, Species> speciesBySurveyCode; + + protected final Map<String, SpeciesProtocol> speciesProtocolBySurveyCode; + + // set of not found species already added in the errors + protected final Set<Species> speciesNotRecognized = new HashSet<>(); + + //set of species not in the protocol + protected final Set<Species> speciesNotInProtocol = new HashSet<>(); + + // set of species without lengthstep already added in the errors + protected final Set<Species> speciesInProtocolButWithoutLengthStepPmfmId = new HashSet<>(); + + private final List<BigfinDataRow> speciesRows = new ArrayList<>(); + + private final List<BigfinDataRow> speciesBatchRows = new ArrayList<>(); + + private final BigfinImportResult bigfinImportResult; + + private final BatchContainer<SpeciesBatch> rootSpeciesBatch; + + private final Map<Species, Caracteristic> lengthStepPmfmBySpecies; + + public BigfinImportContext(File importFile, + FishingOperation operation, + CatchBatch catchBatch, + Map<Sign, CaracteristicQualitativeValue> signsToCaracteristicValue, + Map<String, Species> speciesBySurveyCode, + Map<String, SpeciesProtocol> speciesProtocolBySurveyCode, + BatchContainer<SpeciesBatch> rootSpeciesBatch) { + this.rootSpeciesBatch = rootSpeciesBatch; + this.bigfinImportResult = new BigfinImportResult(importFile); + this.operation = operation; + this.catchBatch = catchBatch; + this.signsToCaracteristicValue = signsToCaracteristicValue; + this.speciesBySurveyCode = speciesBySurveyCode; + this.speciesProtocolBySurveyCode = speciesProtocolBySurveyCode; + this.lengthStepPmfmBySpecies = new HashMap<>(); + } + + public Caracteristic getLengthStepPmfm(Species species, PersistenceService persistenceService) { + Caracteristic caracteristic = lengthStepPmfmBySpecies.get(species); + if (caracteristic == null) { + + SpeciesProtocol speciesProtocol = getSpeciesProtocol(species); + caracteristic = persistenceService.getCaracteristic(Integer.parseInt(speciesProtocol.getLengthStepPmfmId())); + lengthStepPmfmBySpecies.put(species, caracteristic); + + } + return caracteristic; + } + + public boolean isNoError() { + return bigfinImportResult.getErrors().isEmpty(); + } + + public void addRowToProcess(BigfinDataRow bigfinDataRow) { + + if (bigfinDataRow.getSpeciesOrSpeciesBatch().isSpecies()) { + speciesRows.add(bigfinDataRow); + } else { + speciesBatchRows.add(bigfinDataRow); + } + + } + + public Map<Integer, SpeciesBatch> getSpeciesBatchesById() { + return SpeciesBatchs.getAllSpeciesBatchesById(this.rootSpeciesBatch); + } + + public Multimap<Species, SpeciesBatch> getRootSpeciesBatchBySpecies() { + Multimap<Species, SpeciesBatch> batchesBySpecies = Multimaps.index(rootSpeciesBatch.getChildren(), new Function<SpeciesBatch, Species>() { + @Override + public Species apply(SpeciesBatch input) { + return input.getSpecies(); + } + }); + return batchesBySpecies; + } + + public Multimap<Species, BigfinDataRow> getSpeciesRowsBySpecies() { + + // separate the imported rows by species + Multimap<Species, BigfinDataRow> rowsBySpecies = Multimaps.index(speciesRows, new Function<BigfinDataRow, Species>() { + @Override + public Species apply(BigfinDataRow bigfinDataRow) { + return bigfinDataRow.getSpeciesOrSpeciesBatch().getSpecies(); + } + }); + + return rowsBySpecies; + } + + public Multimap<SpeciesBatch, BigfinDataRow> getSpeciesBatchRowsBySpeciesBatch() { + + Multimap<SpeciesBatch, BigfinDataRow> rowsBySpeciesBatch = Multimaps.index(speciesBatchRows, new Function<BigfinDataRow, SpeciesBatch>() { + @Override + public SpeciesBatch apply(BigfinDataRow input) { + return input.getSpeciesOrSpeciesBatch().getBatch(); + } + }); + + return rowsBySpeciesBatch; + + } + + public boolean isStationFound(BigfinDataRow bean) { + + String station = bean.getStation(); + Date date = bean.getDate(); + boolean result = station != null + && station.equals(operation.getStationNumber()) + && date != null + && DateUtils.isSameDay(date, operation.getGearShootingStartDate()); + return result; + + } + + public SpeciesProtocol getSpeciesProtocol(Species species) { + + String code = species.getSurveyCode(); + if (StringUtils.isBlank(code)) { + code = species.getReferenceTaxonId().toString(); + } + SpeciesProtocol speciesProtocol = speciesProtocolBySurveyCode.get(code); + return speciesProtocol; + + } + + public boolean checkRow(BigfinDataRow bigfinDataRow) { + + // check if the station is the one of the operation + // and do not check again a species that has not been recognized before + boolean stationFound = isStationFound(bigfinDataRow); + + boolean canBeAdd = false; + + if (stationFound) { + + SpeciesOrSpeciesBatch speciesOrspeciesBatch = bigfinDataRow.getSpeciesOrSpeciesBatch(); + + if (speciesOrspeciesBatch.isSpecies()) { + + // do some checks on the given species + + checkSizeIsDefined(bigfinDataRow); + + Species species = speciesOrspeciesBatch.getSpecies(); + + boolean speciesIsKnown = checkSpeciesIsKnown(species); + + if (speciesIsKnown) { + + boolean speciesProtocoleIsSafe = checkSpeciesProtocol(species); + + if (speciesProtocoleIsSafe) { + + canBeAdd = true; + + } + } + + } else { + + // do some checks on the given species batch + + boolean speciesBatchIsLeaf = checkSpeciesBatchIsLeaf(bigfinDataRow); + + if (speciesBatchIsLeaf) { + + canBeAdd = true; + + } + } + + } else { + if (log.isInfoEnabled()) { + log.info("Station is not matching for record: " + bigfinDataRow.getRecordId()); + } + } + + return canBeAdd; + } + + public boolean checkSpeciesBatchIsLeaf(BigfinDataRow bigfinDataRow) { + + Preconditions.checkArgument(bigfinDataRow.getSpeciesOrSpeciesBatch().isBatch()); + + + SpeciesBatch batch = bigfinDataRow.getSpeciesOrSpeciesBatch().getBatch(); + boolean speciesBatchIsLeaf = batch.isChildBatchsEmpty(); + + if (!speciesBatchIsLeaf) { + + String warning = t("tutti.service.bigfinImport.warning.speciesBatch.tooCategorized", batch.getId()); + if (log.isWarnEnabled()) { + log.warn(warning); + } + bigfinImportResult.addWarning(warning); + } + + return speciesBatchIsLeaf; + + } + + public void checkSizeIsDefined(BigfinDataRow bigfinDataRow) { + + Preconditions.checkArgument(bigfinDataRow.getSpeciesOrSpeciesBatch().isSpecies()); + + if (bigfinDataRow.getSize() == null) { + String error = t("tutti.service.bigfinImport.error.szClass.unknwon", bigfinDataRow.getRecordId()); + if (log.isErrorEnabled()) { + log.error(error); + } + bigfinImportResult.addError(error); + } + + } + + public boolean checkSpeciesIsKnown(Species species) { + + boolean speciesIsKnown = true; + + if (species.getId() == null) { + + // bloquer tout si un "species" ne match pas le référentiel de Tutti : lister dans ce cas les codes non reconnus + if (speciesNotRecognized.add(species)) { + String error = t("tutti.service.bigfinImport.error.species.not.found", species.getExternalCode()); + if (log.isErrorEnabled()) { + log.error(error); + } + bigfinImportResult.addError(error); + } + + speciesIsKnown = false; + + } + + return speciesIsKnown; + + } + + public boolean checkSpeciesProtocol(Species species) { + + boolean speciesProtocoleIsSafe = true; + + String speciesLabel = getSpeciesLabel(species); + + SpeciesProtocol speciesProtocol = getSpeciesProtocol(species); + + // On n'importe pas les espèces non présentes dans le protocole et + // on liste les espèces/catégorisées non importées pour aider l'utilisateur + // à identifier le problème et on fait l'import des autres + if (speciesProtocol == null) { + + speciesProtocoleIsSafe = false; + + if (speciesNotInProtocol.add(species)) { + + String error = t("tutti.service.bigfinImport.warning.species.notInProtocol", speciesLabel); + if (log.isWarnEnabled()) { + log.warn(error); + } + bigfinImportResult.addWarning(error); + } + + } else { + + if (speciesProtocol.getLengthStepPmfmId() == null) { + + speciesProtocoleIsSafe = false; + + if (speciesInProtocolButWithoutLengthStepPmfmId.add(species)) { + // bloquer toute espèce reconnue du protocole mais qui n'a pas de méthode de mesure + String error = t("tutti.service.bigfinImport.error.species.without.lengthstep", speciesLabel); + if (log.isErrorEnabled()) { + log.error(error); + } + bigfinImportResult.addError(error); + } + } + } + + return speciesProtocoleIsSafe; + + } + + public BigfinImportResult getResult() { + return bigfinImportResult; + } + + protected String getSpeciesLabel(Species species) { + String speciesLabel = species.getSurveyCode(); + if (StringUtils.isBlank(speciesLabel)) { + speciesLabel = species.getRefTaxCode(); + } + return speciesLabel; + } + + public Species getSpeciesWithSurveyCode(Species species) { + String refTaxCode = species.getRefTaxCode(); + Species speciesWithSurveyCode = speciesBySurveyCode.get(refTaxCode); + return speciesWithSurveyCode; + } +} \ No newline at end of file -- To stop receiving notification emails like this one, please contact codelutin.com SCM administrator <admin+scm@codelutin.com>.
This is an automated email from the git hooks/post-receive script. New commit to branch develop in repository tutti. See http://git.codelutin.com/tutti.git commit f11ac3ab84c28f8550457d523c752d2c8a4ec876 Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:13:42 2015 +0100 suppression des vieux signes --- .../fr/ifremer/tutti/service/bigfin/Signs.java | 216 --------------------- 1 file changed, 216 deletions(-) diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/Signs.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/Signs.java deleted file mode 100644 index babbe08..0000000 --- a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/Signs.java +++ /dev/null @@ -1,216 +0,0 @@ -package fr.ifremer.tutti.service.bigfin; - -/* - * #%L - * Tutti :: Service - * %% - * Copyright (C) 2012 - 2014 Ifremer - * %% - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as - * published by the Free Software Foundation, either version 3 of the - * License, or (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public - * License along with this program. If not, see - * <http://www.gnu.org/licenses/gpl-3.0.html>. - * #L% - */ - -import fr.ifremer.adagio.core.dao.referential.pmfm.PmfmId; -import fr.ifremer.adagio.core.dao.referential.pmfm.QualitativeValueId; -import fr.ifremer.tutti.persistence.entities.referential.Caracteristic; -import fr.ifremer.tutti.persistence.entities.referential.CaracteristicQualitativeValue; -import fr.ifremer.tutti.persistence.entities.referential.CaracteristicQualitativeValues; - -import java.util.Map; - -/** - * Created on 5/14/14. - * - * @author Tony Chemit <chemit@codelutin.com> - * @since 3.4.2 - */ -public enum Signs { - - // sexe, M= mâle ; F = femelle ; not set = non sexé - NOT_SEXED("NOT SET") { - @Override - public Integer getCategory() { - return PmfmId.SEX.getValue(); - } - - @Override - public Integer getQualitativeValueId() { - return QualitativeValueId.NON_SEXED_SEX.getValue(); - } - - @Override - public boolean isNullEquivalent() { - return true; - } - }, - MALE("M") { - @Override - public Integer getCategory() { - return PmfmId.SEX.getValue(); - } - - @Override - public Integer getQualitativeValueId() { - return QualitativeValueId.SEX_MALE.getValue(); - } - - @Override - public boolean isNullEquivalent() { - return false; - } - }, - FEMALE("F") { - @Override - public Integer getCategory() { - return PmfmId.SEX.getValue(); - } - - @Override - public Integer getQualitativeValueId() { - return QualitativeValueId.SEX_FEMALE.getValue(); - } - - @Override - public boolean isNullEquivalent() { - return false; - } - }, - - // classe de taille, 1 = petit ; 2 = gros ; 0 = pas de classe de taille (saisie libre donc risque fort de mauvaise saisie) - NOT_SIZED("0") { - @Override - public Integer getCategory() { - return PmfmId.SIZE_CATEGORY.getValue(); - } - - @Override - public Integer getQualitativeValueId() { - return QualitativeValueId.UNSORTED.getValue(); - } - - @Override - public boolean isNullEquivalent() { - return true; - } - - }, - SMALL("1") { - @Override - public Integer getCategory() { - return PmfmId.SIZE_CATEGORY.getValue(); - } - - @Override - public Integer getQualitativeValueId() { - return QualitativeValueId.SIZE_SMALL.getValue(); - } - - @Override - public boolean isNullEquivalent() { - return false; - } - }, - BIG("2") { - @Override - public Integer getCategory() { - return PmfmId.SIZE_CATEGORY.getValue(); - } - - @Override - public Integer getQualitativeValueId() { - return QualitativeValueId.SIZE_BIG.getValue(); - } - - @Override - public boolean isNullEquivalent() { - return false; - } - }, - - // vrac / hors vrac - VRAC("") { - @Override - public Integer getCategory() { - return PmfmId.SORTED_UNSORTED.getValue(); - } - - @Override - public Integer getQualitativeValueId() { - return QualitativeValueId.SORTED_VRAC.getValue(); - } - - @Override - public boolean isNullEquivalent() { - return false; - } - }, - HORS_VRAC("HV") { - @Override - public Integer getCategory() { - return PmfmId.SORTED_UNSORTED.getValue(); - } - - @Override - public Integer getQualitativeValueId() { - return QualitativeValueId.SORTED_HORS_VRAC.getValue(); - } - - @Override - public boolean isNullEquivalent() { - return false; - } - }; - - private String sign; - - Signs(String sign) { - this.sign = sign; - } - - public String getSign() { - return sign; - } - - public static Signs getSign(String sign) { - Signs result = getSign(sign, null); - return result; - } - - public static Signs getSign(String sign, Integer categoryId) { - Signs result = null; - for (Signs s : values()) { - if (s.sign.equals(sign) && - (categoryId == null || s.getCategory().equals(categoryId))) { - result = s; - break; - } - } - return result; - } - - public abstract Integer getCategory(); - - public abstract Integer getQualitativeValueId(); - - // if true, can use this value in the import file to replace a skipped category - public abstract boolean isNullEquivalent(); - - public void registerSign(Caracteristic caracteristic, - Map<Signs, CaracteristicQualitativeValue> map) { - Integer valueId = getQualitativeValueId(); - CaracteristicQualitativeValue result = CaracteristicQualitativeValues.getQualitativeValue(caracteristic, valueId); - map.put(this, result); - } -} -- To stop receiving notification emails like this one, please contact codelutin.com SCM administrator <admin+scm@codelutin.com>.
This is an automated email from the git hooks/post-receive script. New commit to branch develop in repository tutti. See http://git.codelutin.com/tutti.git commit 8e2435c767b55bf3171a1ea9fe98fe8b9272234d Author: Tony CHEMIT <chemit@codelutin.com> Date: Tue Feb 3 19:15:39 2015 +0100 réusinage du code d'import et prise en compte du nouveau mode d'import (en indiquant le lot plutot que l'espèce carégorisée) --- .../tutti/service/bigfin/BigfinImportService.java | 366 +++++++++------------ .../resources/i18n/tutti-service_en_GB.properties | 1 + .../resources/i18n/tutti-service_fr_FR.properties | 3 +- 3 files changed, 159 insertions(+), 211 deletions(-) diff --git a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinImportService.java b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinImportService.java index eb2e061..3ccb390 100644 --- a/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinImportService.java +++ b/tutti-service/src/main/java/fr/ifremer/tutti/service/bigfin/BigfinImportService.java @@ -53,11 +53,15 @@ import fr.ifremer.tutti.persistence.entities.referential.Species; import fr.ifremer.tutti.service.AbstractTuttiService; import fr.ifremer.tutti.service.PersistenceService; import fr.ifremer.tutti.service.TuttiServiceContext; +import fr.ifremer.tutti.service.bigfin.csv.BigfinDataRow; +import fr.ifremer.tutti.service.bigfin.csv.BigfinDataRowModel; +import fr.ifremer.tutti.service.bigfin.signs.Sex; +import fr.ifremer.tutti.service.bigfin.signs.Sign; +import fr.ifremer.tutti.service.bigfin.signs.Size; +import fr.ifremer.tutti.service.bigfin.signs.VracHorsVrac; import fr.ifremer.tutti.util.Weights; import org.apache.commons.collections4.CollectionUtils; -import org.apache.commons.io.IOUtils; import org.apache.commons.lang3.StringUtils; -import org.apache.commons.lang3.time.DateUtils; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.nuiton.csv.Import; @@ -65,14 +69,13 @@ import org.nuiton.csv.ImportRuntimeException; import org.nuiton.jaxx.application.ApplicationBusinessException; import java.io.File; +import java.io.IOException; import java.io.Reader; import java.io.Serializable; import java.text.DateFormat; import java.util.ArrayList; import java.util.Collection; -import java.util.Date; import java.util.HashMap; -import java.util.HashSet; import java.util.List; import java.util.Map; import java.util.Set; @@ -91,46 +94,38 @@ public class BigfinImportService extends AbstractTuttiService { protected Caracteristic sizeCaracteristic; - protected Caracteristic genderCaracteristic; + protected Caracteristic sexCaracteristic; - protected Map<Signs, CaracteristicQualitativeValue> signsToCaracteristicValue; - - protected Map<String, SpeciesProtocol> speciesProtocolBySurveyCode; + protected Map<Sign, CaracteristicQualitativeValue> signsToCaracteristicValue; @Override public void setServiceContext(TuttiServiceContext context) { + super.setServiceContext(context); persistenceService = getService(PersistenceService.class); - signsToCaracteristicValue = Maps.newEnumMap(Signs.class); + signsToCaracteristicValue = new HashMap<>(); - { // sorted/unsorted caracteristic - Caracteristic caracteristic = - persistenceService.getSortedUnsortedCaracteristic(); - Signs.VRAC.registerSign(caracteristic, signsToCaracteristicValue); - Signs.HORS_VRAC.registerSign(caracteristic, signsToCaracteristicValue); - } + // sex caracteristic + sexCaracteristic = persistenceService.getSexCaracteristic(); + Sex.NONE.registerSign(sexCaracteristic, signsToCaracteristicValue); + Sex.MALE.registerSign(sexCaracteristic, signsToCaracteristicValue); + Sex.FEMALE.registerSign(sexCaracteristic, signsToCaracteristicValue); - { // size caracteristic - sizeCaracteristic = persistenceService.getSizeCategoryCaracteristic(); - Signs.NOT_SIZED.registerSign(sizeCaracteristic, signsToCaracteristicValue); - Signs.SMALL.registerSign(sizeCaracteristic, signsToCaracteristicValue); - Signs.BIG.registerSign(sizeCaracteristic, signsToCaracteristicValue); - } + // size caracteristic + sizeCaracteristic = persistenceService.getSizeCategoryCaracteristic(); + Size.NOT_SIZED.registerSign(sizeCaracteristic, signsToCaracteristicValue); + Size.SMALL.registerSign(sizeCaracteristic, signsToCaracteristicValue); + Size.BIG.registerSign(sizeCaracteristic, signsToCaracteristicValue); - { // sex caracteristic - genderCaracteristic = persistenceService.getSexCaracteristic(); - Signs.NOT_SEXED.registerSign(genderCaracteristic, signsToCaracteristicValue); - Signs.MALE.registerSign(genderCaracteristic, signsToCaracteristicValue); - Signs.FEMALE.registerSign(genderCaracteristic, signsToCaracteristicValue); - } + // sorted/unsorted caracteristic + Caracteristic vracHorsVracCaracteristic = persistenceService.getSortedUnsortedCaracteristic(); + VracHorsVrac.VRAC.registerSign(vracHorsVracCaracteristic, signsToCaracteristicValue); + VracHorsVrac.HORS_VRAC.registerSign(vracHorsVracCaracteristic, signsToCaracteristicValue); } - public BigfinImportResult importFile(File bigfinFile, FishingOperation operation, CatchBatch catchBatch) { - - Preconditions.checkNotNull(bigfinFile); - Preconditions.checkArgument(bigfinFile.exists(), "Bigfin file " + bigfinFile + " does not exist."); + private BigfinImportContext prepareImportContext(File importFile, FishingOperation operation, CatchBatch catchBatch) { TuttiProtocol protocol = persistenceService.getProtocol(); @@ -139,7 +134,6 @@ public class BigfinImportService extends AbstractTuttiService { } List<Species> allReferentSpecies = persistenceService.getAllReferentSpecies(); - List<Species> allSpeciesWithSurveyCode = persistenceService.getReferentSpeciesWithSurveyCode(allReferentSpecies); Map<String, Species> speciesBySurveyCode = Maps.newTreeMap(); for (Species species : allReferentSpecies) { @@ -147,12 +141,13 @@ public class BigfinImportService extends AbstractTuttiService { if (StringUtils.isNotBlank(surveyCode)) { speciesBySurveyCode.put(surveyCode, species); - } else if (species.getRefTaxCode() != null) { + } + if (species.getRefTaxCode() != null) { speciesBySurveyCode.put(species.getRefTaxCode(), species); } } - speciesProtocolBySurveyCode = Maps.newTreeMap(); + Map<String, SpeciesProtocol> speciesProtocolBySurveyCode = Maps.newTreeMap(); for (SpeciesProtocol speciesProtocol : protocol.getSpecies()) { if (StringUtils.isNotBlank(speciesProtocol.getSpeciesSurveyCode())) { speciesProtocolBySurveyCode.put(speciesProtocol.getSpeciesSurveyCode(), speciesProtocol); @@ -164,196 +159,147 @@ public class BigfinImportService extends AbstractTuttiService { BatchContainer<SpeciesBatch> rootSpeciesBatch = persistenceService.getRootSpeciesBatch(operation.getId(), false); - // set of not found species already added in the errors - Set<Species> speciesNotRecognized = new HashSet<>(); - //set of species not in the protocol - Set<Species> speciesNotInProtocol = new HashSet<>(); - // set of species without lengthstep alreay added in the errors - Set<Species> speciesInProtocolButWithoutLengthStepPmfmId = new HashSet<>(); - - List<BigfinDataRow> rows = new ArrayList<>(); - BigfinImportResult result = new BigfinImportResult(bigfinFile); - - // load model - BigfinDataRowModel importModel = new BigfinDataRowModel(';', speciesBySurveyCode); - Reader reader = null; - Import<BigfinDataRow> importer = null; - try { - reader = Files.newReader(bigfinFile, Charsets.UTF_8); - importer = Import.newImport(importModel, reader); - - for (BigfinDataRow bean : importer) { - - Species species = bean.getSpecies(); - - // check if the station is the one of the operation - // and do not check again a species that has not been recognized before - String station = bean.getStation(); - Date dt = bean.getDt(); - if (station != null && station.equals(operation.getStationNumber()) - && dt != null && DateUtils.isSameDay(dt, operation.getGearShootingStartDate())) { - - if (bean.getSzClass() == null) { - String error = t("tutti.service.bigfinImport.error.szClass.unknwon", bean.getRecordId()); - if (log.isErrorEnabled()) { - log.error(error); - } - result.addError(error); - } - if (species == null || species.getId() == null) { -// bloquer tout si un "species" ne match pas le référentiel de Tutti : lister dans ce cas les codes non reconnus - if (speciesNotRecognized.add(species)) { - String error = t("tutti.service.bigfinImport.error.species.not.found", species.getExternalCode()); - if (log.isErrorEnabled()) { - log.error(error); - } - result.addError(error); - } + BigfinImportContext bigfinImportContext = new BigfinImportContext(importFile, operation, catchBatch, signsToCaracteristicValue, speciesBySurveyCode, speciesProtocolBySurveyCode, rootSpeciesBatch); + return bigfinImportContext; - } else { - String code = species.getSurveyCode(); - if (StringUtils.isBlank(code)) { - code = species.getReferenceTaxonId().toString(); - } - SpeciesProtocol speciesProtocol = speciesProtocolBySurveyCode.get(code); + } - String speciesLabel = species.getSurveyCode(); - if (StringUtils.isBlank(speciesLabel)) { - speciesLabel = species.getRefTaxCode(); - } + public BigfinImportResult importFile(File importFile, FishingOperation operation, CatchBatch catchBatch) { -// On n'importe pas les espèces non présentes dans le protocole et -// on liste les espèces/catégorisées non importées pour aider l'utilisateur -// à identifier le problème et on fait l'import des autres - if (speciesProtocol == null) { - if (speciesNotInProtocol.add(species)) { - String error = t("tutti.service.bigfinImport.warning.species.notInProtocol", speciesLabel); - if (log.isWarnEnabled()) { - log.warn(error); - } - result.addWarning(error); - } + Preconditions.checkNotNull(importFile); + Preconditions.checkArgument(importFile.exists(), "Bigfin file " + importFile + " does not exist."); - } else if (speciesProtocol.getLengthStepPmfmId() == null) { - if (speciesInProtocolButWithoutLengthStepPmfmId.add(species)) { -// bloquer toute espèce reconnue du protocole mais qui n'a pas de méthode de mesure - String error = t("tutti.service.bigfinImport.error.species.without.lengthstep", speciesLabel); - if (log.isErrorEnabled()) { - log.error(error); - } - result.addError(error); - } - } else { - rows.add(bean); - } + BigfinImportContext importContext = prepareImportContext(importFile, operation, catchBatch); + + BigfinDataRowModel importModel = new BigfinDataRowModel(importContext.speciesBySurveyCode, importContext.getSpeciesBatchesById()); + + try (Reader reader = Files.newReader(importFile, Charsets.UTF_8)) { + + try (Import<BigfinDataRow> importer = Import.newImport(importModel, reader)) { + + for (BigfinDataRow bigfinDataRow : importer) { + + if (log.isInfoEnabled()) { + log.info("Check row: " + bigfinDataRow.getRecordId()); } + boolean rowIsSafe = importContext.checkRow(bigfinDataRow); + + if (rowIsSafe) { + + importContext.addRowToProcess(bigfinDataRow); + } + } + } - } catch (ImportRuntimeException e) { - throw e; + } catch (IOException e) { + throw new ImportRuntimeException("Could not import bigfin data from file " + importFile, e); - } catch (Exception e) { - throw new ImportRuntimeException("Could not import bigfin data from file " + bigfinFile, e); + } + + if (importContext.isNoError()) { + + processSpeciesBatchRows(importContext); + + processSpeciesRows(importContext); + + addFileAsAttachment(importFile, importContext.catchBatch); - } finally { - IOUtils.closeQuietly(importer); - IOUtils.closeQuietly(reader); } - // if no error - if (result.isDone()) { - // get the root batches - List<SpeciesBatch> rootSpeciesBatches = rootSpeciesBatch.getChildren(); - // and separate them by species - Multimap<Species, SpeciesBatch> batchesBySpecies = Multimaps.index(rootSpeciesBatches, new Function<SpeciesBatch, Species>() { - @Override - public Species apply(SpeciesBatch input) { - return input.getSpecies(); - } - }); + BigfinImportResult result = importContext.getResult(); + return result; - // separate the imported rows by species - Multimap<Species, BigfinDataRow> rowsBySpecies = Multimaps.index(rows, new Function<BigfinDataRow, Species>() { - @Override - public Species apply(BigfinDataRow bigfinDataRow) { - return bigfinDataRow.getSpecies(); - } - }); - - SampleCategoryModel sampleCategoryModel = context.getSampleCategoryModel(); - List<Integer> samplingOrder = sampleCategoryModel.getSamplingOrder(); - - List<Integer> pmfmIds = new ArrayList<>(); - pmfmIds.add(PmfmId.SORTED_UNSORTED.getValue()); - List<Function<BigfinDataRow, Signs>> functions = new ArrayList<>(); - - // put the size and order in the right order - for (Integer categoryId : samplingOrder) { - if (PmfmId.SIZE_CATEGORY.getValue().equals(categoryId)) { - pmfmIds.add(categoryId); - functions.add(new Function<BigfinDataRow, Signs>() { - @Override - public Signs apply(BigfinDataRow bigfinDataRow) { - Signs result = bigfinDataRow.getSzClass(); - return result; - } - }); - - } else if (PmfmId.SEX.getValue().equals(categoryId)) { - pmfmIds.add(categoryId); - functions.add(new Function<BigfinDataRow, Signs>() { - @Override - public Signs apply(BigfinDataRow bigfinDataRow) { - Signs result = bigfinDataRow.getGender(); - return result; - } - }); - } - } + } - List<Category> categories = new ArrayList<>(); - for (int i = 0; i < pmfmIds.size(); i++) { - Category category = new Category(pmfmIds.get(i), i < functions.size() ? functions.get(i) : null); - categories.add(category); - } - // for each species imported - for (Species species : rowsBySpecies.keySet()) { + private void processSpeciesBatchRows(BigfinImportContext importContext) { - // get the speciesprotocol and its lengthstep pmfm - String code = species.getSurveyCode(); - if (StringUtils.isBlank(code)) { - code = species.getReferenceTaxonId().toString(); - } - SpeciesProtocol speciesProtocol = speciesProtocolBySurveyCode.get(code); - Caracteristic lengthStepPmfm = persistenceService.getCaracteristic(Integer.parseInt(speciesProtocol.getLengthStepPmfmId())); - - // get the rows with the current species and separate them by vrac/hors varc - Collection<BigfinDataRow> speciesRows = rowsBySpecies.get(species); - Multimap<Signs, BigfinDataRow> rowsByVracHorsVrac = - Multimaps.index(speciesRows, new Function<BigfinDataRow, Signs>() { - @Override - public Signs apply(BigfinDataRow bigfinDataRow) { - Signs result = bigfinDataRow.getVracHorsVrac(); - return result; - } - }); + BigfinImportResult result = importContext.getResult(); + + Multimap<SpeciesBatch, BigfinDataRow> speciesBatchRowsBySpeciesBatch = importContext.getSpeciesBatchRowsBySpeciesBatch(); + + for (SpeciesBatch batch : speciesBatchRowsBySpeciesBatch.keySet()) { + + List<SpeciesBatchFrequency> existingSpeciesFrequencies = persistenceService.getAllSpeciesBatchFrequency(batch.getId()); + + Integer deletedNb = persistenceService.countFrequenciesNumber(existingSpeciesFrequencies, false); + + Collection<BigfinDataRow> bigfinDataRows = speciesBatchRowsBySpeciesBatch.get(batch); + + Species species = importContext.getSpeciesWithSurveyCode(batch.getSpecies()); + Caracteristic lengthStepPmfm = importContext.getLengthStepPmfm(species, persistenceService); + List<SpeciesBatchFrequency> frequencies = createFrequencies(batch, bigfinDataRows, lengthStepPmfm); + + persistenceService.saveSpeciesBatchFrequency(batch.getId(), frequencies); + + result.incrementNbFrequenciesDeleted(deletedNb != null ? deletedNb : 0); + + Integer importedNb = persistenceService.countFrequenciesNumber(frequencies, false); + result.incrementNbFrequenciesImported(importedNb != null ? importedNb : 0); + + } - Collection<SpeciesBatch> speciesBatches = batchesBySpecies.get(species); - Map<Serializable, SpeciesBatch> speciesBatchByVracHorsVrac = Maps.uniqueIndex(speciesBatches, SpeciesBatchs.GET_SAMPLE_CATEGORY_VALUE); + } + + private void processSpeciesRows(BigfinImportContext importContext) { + + BigfinImportResult result = importContext.getResult(); + + SampleCategoryModel sampleCategoryModel = context.getSampleCategoryModel(); + List<Integer> samplingOrder = sampleCategoryModel.getSamplingOrder(); + + List<Integer> pmfmIds = new ArrayList<>(); + pmfmIds.add(PmfmId.SORTED_UNSORTED.getValue()); + List<Function<BigfinDataRow, Sign>> functions = new ArrayList<>(); - BrowseBatchesParameter commonParameter = new BrowseBatchesParameter(operation, species, - lengthStepPmfm, categories, result); - browseBatchesToAddFrequencies(commonParameter, null, 0, speciesBatchByVracHorsVrac, rowsByVracHorsVrac); + // put the size and order in the right order + for (Integer categoryId : samplingOrder) { + if (PmfmId.SIZE_CATEGORY.getValue().equals(categoryId)) { + pmfmIds.add(categoryId); + functions.add(Size.newExtractValueFunction()); + + } else if (PmfmId.SEX.getValue().equals(categoryId)) { + pmfmIds.add(categoryId); + functions.add(Sex.newExtractValueFunction()); } + } + + List<Category> categories = new ArrayList<>(); + for (int i = 0; i < pmfmIds.size(); i++) { + Category category = new Category(pmfmIds.get(i), i < functions.size() ? functions.get(i) : null); + categories.add(category); + } + + // and separate them by species + Multimap<Species, SpeciesBatch> batchesBySpecies = importContext.getRootSpeciesBatchBySpecies(); + + // separate the imported rows by species + Multimap<Species, BigfinDataRow> rowsBySpecies = importContext.getSpeciesRowsBySpecies(); + + // for each species imported + for (Species species : rowsBySpecies.keySet()) { + + // get the lengthstep pmfm associated with the species + Caracteristic lengthStepPmfm = importContext.getLengthStepPmfm(species, persistenceService); + + // get the rows with the current species and separate them by vrac/hors varc + Collection<BigfinDataRow> speciesRows = rowsBySpecies.get(species); + Multimap<Sign, BigfinDataRow> rowsByVracHorsVrac = Multimaps.index(speciesRows, VracHorsVrac.newExtractValueFunction()); + + // get the existing species batches for the current species and separate them by vrac/hors varc + Collection<SpeciesBatch> speciesBatches = batchesBySpecies.get(species); + Map<Serializable, SpeciesBatch> speciesBatchByVracHorsVrac = Maps.uniqueIndex(speciesBatches, SpeciesBatchs.GET_SAMPLE_CATEGORY_VALUE); + + BrowseBatchesParameter commonParameter = new BrowseBatchesParameter(importContext.operation, species, lengthStepPmfm, categories, result); + browseBatchesToAddFrequencies(commonParameter, null, 0, speciesBatchByVracHorsVrac, rowsByVracHorsVrac); - addFileAsAttachment(bigfinFile, catchBatch); } - return result; } + /** * Go deeper in the batches until it finds the last of gender or size class, then add the frequencies * @@ -367,11 +313,11 @@ public class BigfinImportService extends AbstractTuttiService { SpeciesBatch parentBatch, int depth, Map<Serializable, SpeciesBatch> batchesByCaracteristic, - Multimap<Signs, BigfinDataRow> rowsByCaracteristic) { + Multimap<Sign, BigfinDataRow> rowsByCaracteristic) { Category category = commonParameter.getCategories().get(depth++); - for (Signs caracteristic : rowsByCaracteristic.keySet()) { + for (Sign caracteristic : rowsByCaracteristic.keySet()) { Collection<BigfinDataRow> bigfinDataRows = rowsByCaracteristic.get(caracteristic); // get the batch with the caracteristic @@ -404,7 +350,7 @@ public class BigfinImportService extends AbstractTuttiService { commonParameter.getResult().addWarning(t("tutti.service.bigfinImport.warning.species.tooCategorized", commonParameter.getSpeciesLabel(), sizeCaracteristic.getParameterName(), - genderCaracteristic.getParameterName())); + sexCaracteristic.getParameterName())); } else { // create the frequencies @@ -429,7 +375,7 @@ public class BigfinImportService extends AbstractTuttiService { } else { List<SpeciesBatch> batchChildren = batch.getChildBatchs(); - Multimap<Signs, BigfinDataRow> rowsByNewCaracteristic = Multimaps.index(bigfinDataRows, category.getCategoryValueGetter()); + Multimap<Sign, BigfinDataRow> rowsByNewCaracteristic = Multimaps.index(bigfinDataRows, category.getCategoryValueGetter()); // get the children of the current batch and separate them by caracteristic Map<Serializable, SpeciesBatch> childrenByCaracteristic = new HashMap<>(); @@ -444,7 +390,7 @@ public class BigfinImportService extends AbstractTuttiService { // if all the rows to import have a gender or size (the first category in the list) null equivalent // then ok // else error - Set<Signs> signsSet = rowsByNewCaracteristic.keySet(); + Set<Sign> signsSet = rowsByNewCaracteristic.keySet(); if (signsSet.size() == 1 && signsSet.iterator().next().isNullEquivalent()) {// we can go deeper category = commonParameter.getCategories().get(depth++); rowsByNewCaracteristic = Multimaps.index(bigfinDataRows, category.getCategoryValueGetter()); @@ -455,7 +401,7 @@ public class BigfinImportService extends AbstractTuttiService { commonParameter.getResult().addWarning(t("tutti.service.bigfinImport.warning.species.categoriesSkipped", commonParameter.getSpeciesLabel(), sizeCaracteristic.getParameterName(), - genderCaracteristic.getParameterName() + sexCaracteristic.getParameterName() )); continue; } @@ -481,7 +427,7 @@ public class BigfinImportService extends AbstractTuttiService { protected SpeciesBatch createSpeciesBatch(Species species, FishingOperation operation, Integer categoryId, - Signs signs, + Sign signs, String parentBatchId) { Preconditions.checkArgument(signs.getCategory().equals(categoryId)); @@ -571,9 +517,9 @@ public class BigfinImportService extends AbstractTuttiService { private Integer pmfmId; /** function to get the value of the caracteristic we want to group the batches by (eg size or gender) */ - private Function<BigfinDataRow, Signs> categoryValueGetter; + private Function<BigfinDataRow, Sign> categoryValueGetter; - public Category(Integer pmfmId, Function<BigfinDataRow, Signs> dataGetter) { + public Category(Integer pmfmId, Function<BigfinDataRow, Sign> dataGetter) { this.pmfmId = pmfmId; this.categoryValueGetter = dataGetter; } @@ -582,7 +528,7 @@ public class BigfinImportService extends AbstractTuttiService { return pmfmId; } - public Function<BigfinDataRow, Signs> getCategoryValueGetter() { + public Function<BigfinDataRow, Sign> getCategoryValueGetter() { return categoryValueGetter; } } diff --git a/tutti-service/src/main/resources/i18n/tutti-service_en_GB.properties b/tutti-service/src/main/resources/i18n/tutti-service_en_GB.properties index 1639fd0..2e482a1 100644 --- a/tutti-service/src/main/resources/i18n/tutti-service_en_GB.properties +++ b/tutti-service/src/main/resources/i18n/tutti-service_en_GB.properties @@ -69,6 +69,7 @@ tutti.service.bigfinImport.warning.species.categoriesSkipped= tutti.service.bigfinImport.warning.species.categorySkipped= tutti.service.bigfinImport.warning.species.notInProtocol= tutti.service.bigfinImport.warning.species.tooCategorized= +tutti.service.bigfinImport.warning.speciesBatch.tooCategorized= tutti.service.bigfinimport.error.no.protocol= tutti.service.compressZipFile.error= tutti.service.context.serviceInstanciation.error= diff --git a/tutti-service/src/main/resources/i18n/tutti-service_fr_FR.properties b/tutti-service/src/main/resources/i18n/tutti-service_fr_FR.properties index 22d5621..cdde34a 100644 --- a/tutti-service/src/main/resources/i18n/tutti-service_fr_FR.properties +++ b/tutti-service/src/main/resources/i18n/tutti-service_fr_FR.properties @@ -52,12 +52,13 @@ tutti.report.step.load.fishingOperation=Charger le trait sélectionné tutti.service.bigfin.import.attachment.comment=Import Bigfin du %s tutti.service.bigfinImport.error.species.not.found=L'espèce '<strong>%s</strong>' est inconnue tutti.service.bigfinImport.error.species.without.lengthstep=L'espèce '<strong>%s</strong>' n'a pas de classe de taille associée dans le protocole. -tutti.service.bigfinImport.error.szClass.unknwon=Ligne <i>%s</i>, code inconnu (doit être 0, 1 ou 2) +tutti.service.bigfinImport.error.szClass.unknwon=Ligne <i>%s</i>, catégorie de taille inconnue (doit être 0, 1 ou 2) tutti.service.bigfinImport.warning.species.batch.frequenciesOnHigherLevel=Le lot de '<strong>%1s / %2s</strong>' a déjà des mensurations tutti.service.bigfinImport.warning.species.categoriesSkipped=L'espèce '<strong>%1s</strong>' a été catégorisée sans les catégories '<strong>%2s</strong>' et '<strong>%3s</strong>' tutti.service.bigfinImport.warning.species.categorySkipped=L'espèce '<strong>%1s</strong>' a été catégorisée sans la catégorie '<strong>%2s</strong>' tutti.service.bigfinImport.warning.species.notInProtocol=L'espèce '<strong>%1s</strong>' n'est pas présente dans le protocole tutti.service.bigfinImport.warning.species.tooCategorized=L'espèce '<strong>%1s</strong>' est trop catégorisée (pas limitée à '<strong>%2s</strong>' et '<strong>%3s</strong>') +tutti.service.bigfinImport.warning.speciesBatch.tooCategorized=Le lot '<strong>%1s</strong>' contient des sous catégories, on ne peut pas y ajouter des mensurations. tutti.service.bigfinimport.error.no.protocol=Impossible de faire un import Bigfin sans protocol. tutti.service.compressZipFile.error=Erreur lors de la compression du dossier %1s dans le fichier %2s tutti.service.context.serviceInstanciation.error=Erreur lors de l'instanciation du service %s -- To stop receiving notification emails like this one, please contact codelutin.com SCM administrator <admin+scm@codelutin.com>.
This is an automated email from the git hooks/post-receive script. New commit to branch develop in repository tutti. See http://git.codelutin.com/tutti.git commit e33b29ed09fcdcd1f897d9b3171bd54e260be846 Author: Tony CHEMIT <chemit@codelutin.com> Date: Wed Feb 4 11:34:03 2015 +0100 use codelutinpom wagon-m-p version --- pom.xml | 1 - 1 file changed, 1 deletion(-) diff --git a/pom.xml b/pom.xml index caf6f57..9f5e4af 100644 --- a/pom.xml +++ b/pom.xml @@ -887,7 +887,6 @@ <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>wagon-maven-plugin</artifactId> - <version>1.0-beta-4</version> <executions> <execution> <id>deploy tutti-application.properties</id> -- To stop receiving notification emails like this one, please contact codelutin.com SCM administrator <admin+scm@codelutin.com>.
This is an automated email from the git hooks/post-receive script. New commit to branch develop in repository tutti. See http://git.codelutin.com/tutti.git commit 714ca74249a7e13f8968504fbc586789a5dcde8c Merge: e33b29e 8e2435c Author: Tony CHEMIT <chemit@codelutin.com> Date: Wed Feb 4 11:34:39 2015 +0100 fixes #6471 .../persistence/entities/data/SpeciesBatchs.java | 20 ++ .../tutti/service/bigfin/BigfinDataRowModel.java | 138 -------- .../tutti/service/bigfin/BigfinImportContext.java | 357 ++++++++++++++++++++ .../tutti/service/bigfin/BigfinImportService.java | 366 +++++++++------------ .../fr/ifremer/tutti/service/bigfin/Signs.java | 216 ------------ .../service/bigfin/{ => csv}/BigfinDataRow.java | 75 ++--- .../service/bigfin/csv/BigfinDataRowModel.java | 231 +++++++++++++ .../service/bigfin/csv/SpeciesOrSpeciesBatch.java | 42 +++ .../fr/ifremer/tutti/service/bigfin/signs/Sex.java | 109 ++++++ .../ifremer/tutti/service/bigfin/signs/Sign.java | 27 ++ .../ifremer/tutti/service/bigfin/signs/Size.java | 111 +++++++ .../tutti/service/bigfin/signs/VracHorsVrac.java | 92 ++++++ .../resources/i18n/tutti-service_en_GB.properties | 1 + .../resources/i18n/tutti-service_fr_FR.properties | 3 +- 14 files changed, 1186 insertions(+), 602 deletions(-) -- To stop receiving notification emails like this one, please contact codelutin.com SCM administrator <admin+scm@codelutin.com>.
participants (1)
-
codelutin.com scm