- Echobase-commits - mailman.cloud.codelutin.com

r174 - trunk/src/doc/reunions
by tchemit＠users.forge.codelutin.com Dec. 13, 2011

Dec. 13, 2011

Author: tchemit Date: 2011-12-13 15:28:19 +0100 (Tue, 13 Dec 2011) New Revision: 174 Url: http://forge.codelutin.com/repositories/revision/echobase/174 Log: update reunions docs Added: trunk/src/doc/reunions/reunion-2011-12-12.txt trunk/src/doc/reunions/reunion-2011-12-14.txt Modified: trunk/src/doc/reunions/reunion-2011-11-03.txt trunk/src/doc/reunions/reunion-2011-11-15.txt trunk/src/doc/reunions/reunion-2011-12-07.txt Modified: trunk/src/doc/reunions/reunion-2011-11-03.txt =================================================================== --- trunk/src/doc/reunions/reunion-2011-11-03.txt 2011-12-13 14:19:07 UTC (rev 173) +++ trunk/src/doc/reunions/reunion-2011-11-03.txt 2011-12-13 14:28:19 UTC (rev 174) @@ -3,7 +3,7 @@ Jeudi 03-11-2011 (Ifremer) -Présents : Matthieu, Vincent, Benjamin, Tony +Présents : Mathieu, Vincent, Benjamin, Tony Délais ------ @@ -48,7 +48,7 @@ ---------------------- Pour le moment, on ne fait que le premier mode (dit expert), le second mode -paraît pas assez réaliste selon Matthieu (il sera mieux d'utiliser +paraît pas assez réaliste selon Mathieu (il sera mieux d'utiliser Libre-Office qui gère facilement les jointures). - Un requête est caractérisée par Modified: trunk/src/doc/reunions/reunion-2011-11-15.txt =================================================================== --- trunk/src/doc/reunions/reunion-2011-11-15.txt 2011-12-13 14:19:07 UTC (rev 173) +++ trunk/src/doc/reunions/reunion-2011-11-15.txt 2011-12-13 14:28:19 UTC (rev 174) @@ -3,7 +3,7 @@ Lundi 15-11-2011 (Ifremer) -Présents : Matthieu, Vincent, Benjamin, Tony +Présents : Mathieu, Vincent, Benjamin, Tony Planning -------- Modified: trunk/src/doc/reunions/reunion-2011-12-07.txt =================================================================== --- trunk/src/doc/reunions/reunion-2011-12-07.txt 2011-12-13 14:19:07 UTC (rev 173) +++ trunk/src/doc/reunions/reunion-2011-12-07.txt 2011-12-13 14:28:19 UTC (rev 174) @@ -3,7 +3,7 @@ Lundi 07-12-2011 (Ifremer) -Présents : Matthieu, Vincent, Benjamin, Tony +Présents : Mathieu, Vincent, Benjamin, Tony Planning -------- Added: trunk/src/doc/reunions/reunion-2011-12-12.txt =================================================================== --- trunk/src/doc/reunions/reunion-2011-12-12.txt (rev 0) +++ trunk/src/doc/reunions/reunion-2011-12-12.txt 2011-12-13 14:28:19 UTC (rev 174) @@ -0,0 +1,221 @@ +projet echobase +--------------- + +Lundi 12-12-2011 (Ifremer) + +Présents : Mathieu, Benjamin et Tony + +Planning +-------- + +- finaliser les imports du modèle acoustique +- finaliser le modèle pêche + +Import acoustique +----------------- + +Référentiel acoustique +====================== + +Il reste quelques référentiel à revoir pour la partie acoustique : + +- Echotype (utiliser la version envoyée par mathieu) OK +- Species (nouvelle version à intégrer) OK +- VesselType ? A fournir par Mathieu +- DataType : il manque le datumType (tous en Double sauf Formula et ReferenceXXX en String) OK +- DataMetada (sur les nouvelles entrées il manque des références vers cellMethod et referenceDatum) c'est normal me dit Mathieu OK + +D'autres seront vus lors du traitement de la partie pêche : + +- Localisation +- Station +- SpeciesCategory + +? CategoryRef + +Données thématiques +=================== + +Voyage +~~~~~~ + +- 1 voyage par ligne de la table CAMPAGNE +- Pour relier les echotypes aux voyage, on utilise le fichier csv fournit par mathieu OK + +Transit +~~~~~~~ + +- 1 Transit par voyage (donc 1 par ligne de la table CAMPAGNE) + +Transect +~~~~~~~~ + +- 1 Transect par transit (donc 1 par ligne de la table CAMPAGNE) + +DataAcquisition +~~~~~~~~~~~~~~~ + +- Une entrée par ligne de EI_CONFIGURATION + +DataProcessing +~~~~~~~~~~~~~~ + +- Une entrée par ligne de EI_CONFIGURATION (1 par DataAcquisition) + +Cell +~~~~ + +Un certain de nombre de dataProcessing n'ont pas de région : + +- EIT_0510_38 +- EIT_83A_38 +- EIT_83B_38 +- EIT_84_38 +- EIT_85_38 +- EIT_86_38 +- EIT_87_38 +- EIT_89A_38 +- EIT_89B_38 +- EIT_90_38 +- EIT_91_38 +- EIT_92_38 +- EIT_93_38 +- EIT_94_38 +- EIT_97_38 + +Il faudra directement relier aux Cellules Esdu etElementary. + +Si on en trouve pas de qualité sur une cellule, on utilise alors la qualité de code 0. + +Cell (region) +~~~~~~~~~~~~~ + +Crée à partir du DataProcessing id et de la ligne de POST_STRATE qui correspond + +On prend toutes les lignes tel que +EI_ESU#ID_EI_ESDU = dataProcessingId +EI_ESU#DHTU_ESDU + +Pour chacune de ces lignes on cherche dans + +EIT_VALEUR_ECHOS#ID_EI_DEVIATION =EI_ESU#ID_EI_ESDU +EIT_VALEUR_ECHOS#DHTU_DEVIATION = EI_ESU#DHTU_ESDU +EIT_VALEUR_ECHOS#LIBELLE_DEVIATION + +Puis jointure sur la table SCENARIO + +SCENARIO#DHTU_SCENARIO = EIT_VALEUR_ECHOS#DHTU_DEVIATION +SCENARIO#LIBELLE_ECHO_SCENARIO = EIT_VALEUR_ECHOS#LIBELLE_DEVIATION +SCENARIO#POST_STRATE_SCENARIO +SCENARIO#Type_Image + +--> Ici on est pas assuré de trouver la correspondance, + on boucle sur la date suivante). + +Puis jointure sur la table POST_STRATE + +POST_STRATE#ID_STRATE = SCENARIO#POST_STRATE_SCENARIO +POST_STRATE#Type_Image = SCENARIO#Type_Image + +Si on trouve une telle ligne alors on a une région + +cellName = POST_STRATE#ID_STRATE +cellType = RegionXXX où XXX est la correspondance de TypeImage + +Data (region) +~~~~~~~~~~~~~ + +Une seule donnée : surfaceMetadata ("Surface") +qui provient de la colonne SURFACE de la ligne POST_STRATE de la cellule Region + +Cell (esdu) (lié à une région) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Pour chaque ligne de la table SCENARIO avec SCENARIO#POST_STRATE_SCENARIO = CellRegion#cellName + +On récupere SCENARIO#DHTU_SCENARIO, + +Jointure sur la table EI_ESU + +EI_ESU#DHTU_ESDU = SCENARIO#DHTU_SCENARIO + +La ligne = Cell Esdu + +CellName = EI_ESU#DHTU_ESDU +CellType = "Esdu" + +Critère de qualité EI_ESU#FLAG + +Data (esdu) +~~~~~~~~~~~ + +5 données à partir de la ligne de EI_ESU données de type CellMethod à Start + +- esduLatitudeMetadata ("LatitudeStart") EI_ESU#LATDMC +- esduLongitudeMetadata ("LongitudeStart") EI_ESU#LGDMC +- esduDepthMetadata ("") EI_ESU#SONDE = "DepthRefSurfaceStart" +- esduTimeMetadata ("Time") EI_ESU#DHTU_ESDU Time avec CellMethod=Start +- acousticMetadata ("Acoustic") ? EI_ESU#TOTAL : il s'agit du dataMetadata provenant de EI_CONFIGURATION avec name = DataProcessingId + +La qualité est celle de sa cellule + +Cell (esdu) (sans région) +~~~~~~~~~~~~~~~~~~~~~~~~~ + +On se passe de la table SCENARIO + +Chaque esdu est une ligne de EI_ESU#ID_EI_ESDU = dataProcessingId + +Pour les elementary c'est comme avant. + +Cell (elementary) +~~~~~~~~~~~~~~~~~ + +A partir du dataprocessingId et +EIT_ENERGIES_TRANCHES#ID_EI_ESDU = cellRegion#name +EIT_ENERGIES_TRANCHES#DHTU_DEVIATION = cellEsu#name + +On obtient EIT_ENERGIES_TRANCHES#ID_TRANCHE + +Jointure sur EIT_DESCRIPT_TRANCHE +EIT_DESCRIPT_TRANCHE#ID_TRANCHE = EIT_ENERGIES_TRANCHES#ID_TRANCHE + +pour obtenir EIT_DESCRIPT_TRANCHE#LIBELLE_TRANCHE = nom de la cellule élémentaire + +CellName = EIT_DESCRIPT_TRANCHE#LIBELLE_TRANCHE +CellType = "Elementary" + +Qualité = celle de la cellule esdu + +Data (elementary) +~~~~~~~~~~~~~~~~~ + +4 données : + +- acousticMetadata ("Acoustic") ? EIT_ENERGIES_TRANCHES#ENERGY : il s'agit du dataMetadata provenant de EI_CONFIGURATION avec name = DataProcessingId + +Si le nom de la cellule commence par "S" (type surface) + +- elementaryStartSurfaceDepthMetadata ("DepthRefSurfaceStart") EIT_DESCRIPT_TRANCHE#HMIN_TRANCHE +- elementaryEndSurfaceDepthMetadata ("DepthRefSurfaceEnd") EIT_DESCRIPT_TRANCHE#HMAX_TRANCHE + +Si le nom de la cellule commence par "F" (type Bottom) +- elementaryStartBottomDepthMetadata ("DepthRefBottomStart") EIT_DESCRIPT_TRANCHE#HMIN_TRANCHE +- elementaryEndBottonDepthMetadata ("DepthRefBottomEnd") EIT_DESCRIPT_TRANCHE#HMAX_TRANCHE + +- Et si le nom est autre ? (Total) ne rien ajouter. + +Action +------ + +- Code + - finir l'import des données historiques acoustique + - faire l'import des données depuis l'application + +- Ifremer + - fournir le référentiel VesselType + - finir l'import du module pêche + +Prochaine réunion : + + - Mercredi 14 à 14h30 à l'Ifremer avec Mathieu \ No newline at end of file Property changes on: trunk/src/doc/reunions/reunion-2011-12-12.txt ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Added: trunk/src/doc/reunions/reunion-2011-12-14.txt =================================================================== --- trunk/src/doc/reunions/reunion-2011-12-14.txt (rev 0) +++ trunk/src/doc/reunions/reunion-2011-12-14.txt 2011-12-13 14:28:19 UTC (rev 174) @@ -0,0 +1,23 @@ +projet echobase +--------------- + +Mercredi 14-12-2011 (Ifremer) + +Présents : Mathieu et Tony + +Planning +-------- + +- Finaliser le modèle pêche et commencer ses imports +- Présentation de l'application (déployée sur Démo) + +Action +------ + +- Code + +- Ifremer + +Prochaine réunion : + + - Lundi 19 à 10h00 à l'Ifremer \ No newline at end of file Property changes on: trunk/src/doc/reunions/reunion-2011-12-14.txt ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native

1 0

r173 - in trunk: echobase-entities/src/main/xmi echobase-services/src/main/java/fr/ifremer/echobase/services echobase-services/src/main/java/fr/ifremer/echobase/services/models
by tchemit＠users.forge.codelutin.com Dec. 13, 2011

Dec. 13, 2011

Author: tchemit Date: 2011-12-13 15:19:07 +0100 (Tue, 13 Dec 2011) New Revision: 173 Url: http://forge.codelutin.com/repositories/revision/echobase/173 Log: - move functions to entities model - implements all entities decorators Modified: trunk/echobase-entities/src/main/xmi/echobase.zargo trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DecoratorService.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityCsvModel.java Modified: trunk/echobase-entities/src/main/xmi/echobase.zargo =================================================================== (Binary files differ) Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java 2011-12-13 13:25:38 UTC (rev 172) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java 2011-12-13 14:19:07 UTC (rev 173) @@ -25,6 +25,7 @@ import com.google.common.collect.Lists; import com.google.common.collect.Maps; +import fr.ifremer.echobase.EchoBaseFunctions; import fr.ifremer.echobase.EchoBaseIOUtil; import fr.ifremer.echobase.EchoBaseTechnicalException; import fr.ifremer.echobase.entities.EchoBaseEntityEnum; @@ -33,7 +34,6 @@ import fr.ifremer.echobase.entities.meta.AssociationMeta; import fr.ifremer.echobase.entities.meta.MetaFilenameAware; import fr.ifremer.echobase.entities.meta.TableMeta; -import fr.ifremer.echobase.services.models.CsvModelUtil; import fr.ifremer.echobase.services.models.EntityAssociationCsvModel; import org.apache.commons.io.FileUtils; import org.apache.commons.logging.Log; @@ -199,7 +199,7 @@ EchoBaseEntityEnum target = associationMeta.getTarget(); List<TopiaEntity> targetEntities = getEntities(target); Map<String, TopiaEntity> targetsById = - Maps.uniqueIndex(targetEntities, CsvModelUtil.TO_TOPIAID); + Maps.uniqueIndex(targetEntities, EchoBaseFunctions.TO_TOPIAID); ImportModel<TopiaEntity> model = EntityAssociationCsvModel.newImportModel( @@ -308,7 +308,7 @@ // TopiaDAO<TopiaEntity> targetDAO = getDAO(targetType); List<E> sourceEntities = getEntities(sourceType); Map<String, E> sourcesById = - Maps.uniqueIndex(sourceEntities, CsvModelUtil.TO_TOPIAID); + Maps.uniqueIndex(sourceEntities, EchoBaseFunctions.TO_TOPIAID); try { for (E row : importer) { E entityToSave = sourcesById.get(row.getTopiaId()); Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DecoratorService.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DecoratorService.java 2011-12-13 13:25:38 UTC (rev 172) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DecoratorService.java 2011-12-13 14:19:07 UTC (rev 173) @@ -28,8 +28,62 @@ import com.google.common.collect.Maps; import fr.ifremer.echobase.entities.EchoBaseUser; import fr.ifremer.echobase.entities.ExportQuery; +import fr.ifremer.echobase.entities.data.Cell; +import fr.ifremer.echobase.entities.data.Data; +import fr.ifremer.echobase.entities.data.DataAcquisition; +import fr.ifremer.echobase.entities.data.DataProcessing; +import fr.ifremer.echobase.entities.data.Operation; +import fr.ifremer.echobase.entities.data.OperationGearMetadataValue; +import fr.ifremer.echobase.entities.data.OperationMetadataValue; +import fr.ifremer.echobase.entities.data.Result; +import fr.ifremer.echobase.entities.data.Sample; +import fr.ifremer.echobase.entities.data.SampleData; +import fr.ifremer.echobase.entities.data.Transect; +import fr.ifremer.echobase.entities.data.Transit; import fr.ifremer.echobase.entities.data.Voyage; +import fr.ifremer.echobase.entities.references.AcousticInstrument; +import fr.ifremer.echobase.entities.references.AgeCategory; +import fr.ifremer.echobase.entities.references.AreaOfOperation; +import fr.ifremer.echobase.entities.references.Calibration; +import fr.ifremer.echobase.entities.references.Category; +import fr.ifremer.echobase.entities.references.CategoryMeaning; +import fr.ifremer.echobase.entities.references.CategoryRef; +import fr.ifremer.echobase.entities.references.CategoryType; +import fr.ifremer.echobase.entities.references.CellMethod; +import fr.ifremer.echobase.entities.references.CellType; +import fr.ifremer.echobase.entities.references.DataMetadata; +import fr.ifremer.echobase.entities.references.DataProtocol; +import fr.ifremer.echobase.entities.references.DataQuality; +import fr.ifremer.echobase.entities.references.DataType; +import fr.ifremer.echobase.entities.references.DepthStratum; +import fr.ifremer.echobase.entities.references.Echotype; +import fr.ifremer.echobase.entities.references.EchotypeCategory; +import fr.ifremer.echobase.entities.references.Gear; +import fr.ifremer.echobase.entities.references.GearCaracteristic; +import fr.ifremer.echobase.entities.references.GearMetadata; +import fr.ifremer.echobase.entities.references.GearMetadataValue; +import fr.ifremer.echobase.entities.references.GearType; +import fr.ifremer.echobase.entities.references.Impacte; +import fr.ifremer.echobase.entities.references.Localisation; +import fr.ifremer.echobase.entities.references.MeasureType; +import fr.ifremer.echobase.entities.references.MeasurementMetadata; import fr.ifremer.echobase.entities.references.Mission; +import fr.ifremer.echobase.entities.references.Moment; +import fr.ifremer.echobase.entities.references.OperationMetadata; +import fr.ifremer.echobase.entities.references.OperationType; +import fr.ifremer.echobase.entities.references.ReferenceDatum; +import fr.ifremer.echobase.entities.references.ReferenceDatumType; +import fr.ifremer.echobase.entities.references.ReferencingMethod; +import fr.ifremer.echobase.entities.references.SampleDataType; +import fr.ifremer.echobase.entities.references.SampleType; +import fr.ifremer.echobase.entities.references.SexeCategory; +import fr.ifremer.echobase.entities.references.SizeCategory; +import fr.ifremer.echobase.entities.references.Species; +import fr.ifremer.echobase.entities.references.SpeciesCategory; +import fr.ifremer.echobase.entities.references.Station; +import fr.ifremer.echobase.entities.references.TSParameters; +import fr.ifremer.echobase.entities.references.Vessel; +import fr.ifremer.echobase.entities.references.VesselType; import org.apache.commons.collections.CollectionUtils; import org.nuiton.topia.persistence.TopiaEntity; import org.nuiton.util.decorator.Decorator; @@ -119,17 +173,185 @@ @Override protected void loadDecorators(Locale locale) { - // user decorator + // EchoBaseUser decorator registerJXPathDecorator(locale, EchoBaseUser.class, "${email}$s"); - // mission decorator + // ExportQuery decorator + registerJXPathDecorator(locale, ExportQuery.class, "${name}$s - ${description}$s"); + + // AcousticInstrument decorator + registerJXPathDecorator(locale, AcousticInstrument.class, "${id}$s"); + + // AgeCategory decorator + registerJXPathDecorator(locale, AgeCategory.class, "${name}$s"); + + // AreaOfOperation decorator + registerJXPathDecorator(locale, AreaOfOperation.class, "${name}$s"); + + // Calibration decorator + registerJXPathDecorator(locale, Calibration.class, "${date}$s - acquisition ${aquisitionMethod}$s"); + + // Category decorator + registerJXPathDecorator(locale, Category.class, "${classCode}$s ${categoryValue}$s"); + + // CategoryMeaning decorator + registerJXPathDecorator(locale, CategoryMeaning.class, "${name}$s"); + + // CategoryRef decorator + registerJXPathDecorator(locale, CategoryRef.class, "${genusSpecies}$s"); + + // CategoryType decorator + registerJXPathDecorator(locale, CategoryType.class, "${name}$s"); + + // CellMethod decorator + registerJXPathDecorator(locale, CellMethod.class, "${name}$s"); + + // CellType decorator + registerJXPathDecorator(locale, CellType.class, "${name}$s"); + + // DataMetadata decorator + registerJXPathDecorator(locale, DataMetadata.class, "${name}$s"); + + // DataProtocol decorator + registerJXPathDecorator(locale, DataProtocol.class, "${description}$s [ ${startValidityDate}$s - ${endValidityDate}$s ]"); + + // DataQuality decorator + registerJXPathDecorator(locale, DataQuality.class, "${qualityDataFlagValues}$s - ${flagMeanings}$s"); + + // DataType decorator + registerJXPathDecorator(locale, DataType.class, "${name}$s"); + + // DepthStratum decorator + registerJXPathDecorator(locale, DepthStratum.class, "${id}$s - ${meaning}$s"); + + // Echotype decorator + registerJXPathDecorator(locale, Echotype.class, "${id}$s - ${name}$s"); + + // EchotypeCategory decorator + registerJXPathDecorator(locale, EchotypeCategory.class, "${name}$s"); + + // Gear decorator + registerJXPathDecorator(locale, Gear.class, "${name}$s [ ${startValidityDate}$s - ${endValidityDate}$s ]"); + + // GearCaracteristic decorator + registerJXPathDecorator(locale, GearCaracteristic.class, "${name}$s"); + + // GearMetadata decorator + registerJXPathDecorator(locale, GearMetadata.class, "${name}$s"); + + // GearMetadataValue decorator + registerJXPathDecorator(locale, GearMetadataValue.class, "${dataValue}$s"); + + // GearType decorator + registerJXPathDecorator(locale, GearType.class, "${name}$s"); + + // Impacte decorator + registerJXPathDecorator(locale, Impacte.class, "${measurementUnit}$s - ${species.genusSpecies}"); + + // Localisation decorator + registerJXPathDecorator(locale, Localisation.class, "Latitude <${minLatitude}$s - ${maxLatitude}$s>, Longitude <${minLongitude}$s - ${maxLongitude}$s>"); + + // MeasurementMetadata decorator + registerJXPathDecorator(locale, MeasurementMetadata.class, "${name}$s"); + + // MeasureType decorator + registerJXPathDecorator(locale, MeasureType.class, "${name}$s"); + + // Mission decorator registerJXPathDecorator(locale, Mission.class, "${missionName}$s"); - // voyage decorator + // Moment decorator + registerJXPathDecorator(locale, Moment.class, "${name}$s"); + + // OperationMetadata decorator + registerJXPathDecorator(locale, OperationMetadata.class, "${name}$s"); + + // OperationType decorator + registerJXPathDecorator(locale, OperationType.class, "${name}$s"); + + // ReferenceDatum decorator + registerJXPathDecorator(locale, ReferenceDatum.class, "${id}$s"); + + // ReferenceDatumType decorator + registerJXPathDecorator(locale, ReferenceDatumType.class, "${name}$s"); + + // ReferencingMethod decorator + registerJXPathDecorator(locale, ReferencingMethod.class, "${name}$s"); + + // SampleDataType decorator + registerJXPathDecorator(locale, SampleDataType.class, "${name}$s"); + + // SampleType decorator + registerJXPathDecorator(locale, SampleType.class, "${name}$s - level ${level}$s"); + + // SexeCategory decorator + registerJXPathDecorator(locale, SexeCategory.class, "${name}$s"); + + // SizeCategory decorator + registerJXPathDecorator(locale, SizeCategory.class, "${name}$s"); + + // Species decorator + registerJXPathDecorator(locale, Species.class, "${genusSpecies}$s"); + + //TODO + // SpeciesCategory decorator + registerJXPathDecorator(locale, SpeciesCategory.class, "${species.genusSpecies}$s - ..."); + + // Station decorator + registerJXPathDecorator(locale, Station.class, "${startLatitude}$s - ${startLongitude}$s"); + + //TODO + // TSParameters decorator + registerJXPathDecorator(locale, TSParameters.class, "${}$s"); + + // Vessel decorator + registerJXPathDecorator(locale, Vessel.class, "${name}$s"); + + // VesselType decorator + registerJXPathDecorator(locale, VesselType.class, "${name}$s"); + + // Cell decorator + registerJXPathDecorator(locale, Cell.class, "${name}$s"); + + // Data decorator + registerJXPathDecorator(locale, Data.class, "${dataValue}$s"); + + //TODO + // DataAcquisition decorator + registerJXPathDecorator(locale, DataAcquisition.class, ""); + + //TODO + // DataProcessing decorator + registerJXPathDecorator(locale, DataProcessing.class, ""); + + // Operation decorator + registerJXPathDecorator(locale, Operation.class, "${startDate}$s - ${endDate}$s"); + + // OperationGearMetadataValue decorator + registerJXPathDecorator(locale, OperationGearMetadataValue.class, "${dataValue}"); + + // OperationMetadataValue decorator + registerJXPathDecorator(locale, OperationMetadataValue.class, "${dataValue}$s - moment ${moment.name}"); + + // Result decorator + registerJXPathDecorator(locale, Result.class, "${resultvalue}$s"); + + // Sample decorator + registerJXPathDecorator(locale, Sample.class, "${resultvalue}$s"); + + // SampleData decorator + registerJXPathDecorator(locale, SampleData.class, "${resultvalue}$s"); + + // Transect decorator + registerJXPathDecorator(locale, Transect.class, "${title}$s - Vessel ${vessel}$s"); + + // Transit decorator + registerJXPathDecorator(locale, Transit.class, "${startTime}$s - ${endTime}$s"); + + // Voyage decorator registerJXPathDecorator(locale, Voyage.class, "${voyageName}$s"); - // export query decorator - registerJXPathDecorator(locale, ExportQuery.class, "${name}$s - ${description}$s"); + } } } Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java 2011-12-13 13:25:38 UTC (rev 172) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java 2011-12-13 14:19:07 UTC (rev 173) @@ -26,6 +26,7 @@ import com.google.common.base.Function; import com.google.common.collect.Lists; import com.google.common.collect.Maps; +import fr.ifremer.echobase.EchoBaseFunctions; import fr.ifremer.echobase.EchoBaseTechnicalException; import fr.ifremer.echobase.services.DecoratorService; import org.apache.commons.collections.CollectionUtils; @@ -49,20 +50,12 @@ */ public class CsvModelUtil extends Common { - public static final Function<TopiaEntity, String> TO_TOPIAID = new Function<TopiaEntity, String>() { - - @Override - public String apply(TopiaEntity input) { - return input.getTopiaId(); - } - }; - public static <E extends TopiaEntity> ForeignKeyValue<E> newForeignKeyValue(Class<E> type) { return new ForeignKeyValue<E>(type); } public static <E extends TopiaEntity> ForeignKeyValue<E> newForeignKeyValue(Class<E> type, Collection<E> entitites) { - Map<String, E> universe = Maps.uniqueIndex(entitites, TO_TOPIAID); + Map<String, E> universe = Maps.uniqueIndex(entitites, EchoBaseFunctions.TO_TOPIAID); return newForeignKeyValue(type, universe); } @@ -109,7 +102,6 @@ String[] ids = value.split("\\|"); for (String id : ids) { -// E association = ObjectUtil.newInstance(entityType); E association = universe.get(id); association.setTopiaId(id); result.add(association); Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityCsvModel.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityCsvModel.java 2011-12-13 13:25:38 UTC (rev 172) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityCsvModel.java 2011-12-13 14:19:07 UTC (rev 173) @@ -248,6 +248,4 @@ TopiaEntity.TOPIA_ID ); } - - } \ No newline at end of file

1 0

r172 - in trunk/echobase-entities/src/main: java/fr/ifremer/echobase resources/i18n xmi
by tchemit＠users.forge.codelutin.com Dec. 13, 2011

Dec. 13, 2011

Author: tchemit Date: 2011-12-13 14:25:38 +0100 (Tue, 13 Dec 2011) New Revision: 172 Url: http://forge.codelutin.com/repositories/revision/echobase/172 Log: fix property names move functions to this package Added: trunk/echobase-entities/src/main/java/fr/ifremer/echobase/EchoBaseFunctions.java Modified: trunk/echobase-entities/src/main/resources/i18n/echobase-entities_fr_FR.properties trunk/echobase-entities/src/main/xmi/echobase.zargo Added: trunk/echobase-entities/src/main/java/fr/ifremer/echobase/EchoBaseFunctions.java =================================================================== --- trunk/echobase-entities/src/main/java/fr/ifremer/echobase/EchoBaseFunctions.java (rev 0) +++ trunk/echobase-entities/src/main/java/fr/ifremer/echobase/EchoBaseFunctions.java 2011-12-13 13:25:38 UTC (rev 172) @@ -0,0 +1,190 @@ +/* + * #%L + * EchoBase :: Entities + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ +package fr.ifremer.echobase; + +import com.google.common.base.Function; +import fr.ifremer.echobase.entities.data.DataProcessing; +import fr.ifremer.echobase.entities.data.Voyage; +import fr.ifremer.echobase.entities.references.AcousticInstrument; +import fr.ifremer.echobase.entities.references.AreaOfOperation; +import fr.ifremer.echobase.entities.references.CellMethod; +import fr.ifremer.echobase.entities.references.DataQuality; +import fr.ifremer.echobase.entities.references.DataType; +import fr.ifremer.echobase.entities.references.DepthStratum; +import fr.ifremer.echobase.entities.references.EchotypeCategory; +import fr.ifremer.echobase.entities.references.Mission; +import fr.ifremer.echobase.entities.references.ReferenceDatum; +import fr.ifremer.echobase.entities.references.ReferenceDatumType; +import fr.ifremer.echobase.entities.references.Species; +import fr.ifremer.echobase.entities.references.Vessel; +import org.nuiton.topia.persistence.TopiaEntity; + +import java.io.File; +import java.util.Map; + +/** + * Define some usefull function used by application. + * + * @author tchemit <chemit(a)codelutin.com> + * @see Function + * @since 0.2 + */ +public class EchoBaseFunctions { + + public static final Function<Voyage, String> VOYAGE_NAME = new Function<Voyage, String>() { + @Override + public String apply(Voyage input) { + return input.getName().toLowerCase(); + } + }; + + public static final Function<Species, String> SPECIES_BARACOUDA_CODE = new Function<Species, String>() { + @Override + public String apply(Species input) { + return input.getBaracoudaCode(); + } + }; + + public static final Function<Vessel, String> VESSEL_NAME = new Function<Vessel, String>() { + @Override + public String apply(Vessel input) { + return input.getName().toLowerCase(); + } + }; + + public static final Function<AcousticInstrument, String> ACOUSTIC_INSTRUMENT_ID = new Function<AcousticInstrument, String>() { + @Override + public String apply(AcousticInstrument input) { + return input.getId().toLowerCase(); + } + }; + + public static final Function<DepthStratum, String> DEPTH_STRATUM_ID = new Function<DepthStratum, String>() { + @Override + public String apply(DepthStratum input) { + return input.getId(); + } + }; + + public static final Function<EchotypeCategory, String> ECHO_TYPE_CATEGORY_NAME = new Function<EchotypeCategory, String>() { + @Override + public String apply(EchotypeCategory input) { + return input.getName(); + } + }; + + public static final Function<ReferenceDatumType, String> REFERENCE_DATUM_TYPE_NAME = new Function<ReferenceDatumType, String>() { + @Override + public String apply(ReferenceDatumType input) { + return input.getName().toLowerCase(); + } + }; + + public static final Function<ReferenceDatum, String> REFERENCE_DATUM_ID = new Function<ReferenceDatum, String>() { + @Override + public String apply(ReferenceDatum input) { + return input.getId().toLowerCase(); + } + }; + + public static final Function<Mission, String> MISSION_NAME = new Function<Mission, String>() { + @Override + public String apply(Mission input) { + return input.getName().toLowerCase(); + } + }; + + public static final Function<AreaOfOperation, String> AREA_OF_OPERATION_NAME = new Function<AreaOfOperation, String>() { + @Override + public String apply(AreaOfOperation input) { + return input.getName().toLowerCase(); + } + }; + public static final Function<DataType, String> DATA_TYPE_NAME = new Function<DataType, String>() { + @Override + public String apply(DataType input) { + return input.getName().toLowerCase(); + } + }; + public static final Function<CellMethod, String> CELL_METHOD_NAME = new Function<CellMethod, String>() { + @Override + public String apply(CellMethod input) { + return input.getName().toLowerCase(); + } + }; + public static final Function<DataProcessing, String> DATA_PROCESSING_ID = new Function<DataProcessing, String>() { + @Override + public String apply(DataProcessing input) { + return input.getId().toLowerCase(); + } + }; + + public static final Function<DataQuality, Integer> DATA_QUALITY_ID = new Function<DataQuality, Integer>() { + @Override + public Integer apply(DataQuality input) { + return input.getQualityDataFlagValues(); + } + }; + public static final Function<TopiaEntity, String> TO_TOPIAID = new Function<TopiaEntity, String>() { + + @Override + public String apply(TopiaEntity input) { + return input.getTopiaId(); + } + }; + public static final Function<File,String> FILE_NAME = new Function<File, String>() { + @Override + public String apply(File input) { + return input.getName(); + } + }; + + public static Function<Map<String, Object>, String> newRowFunction(final String key) { + + return new Function<Map<String, Object>, String>() { + + @Override + public String apply(Map<String, Object> input) { + String id = input.get(key).toString().toLowerCase(); + return id; + } + }; + } + + public static Function<Map<String, Object>, Integer> newRowIntegerFunction(final String key) { + + return new Function<Map<String, Object>, Integer>() { + + @Override + public Integer apply(Map<String, Object> input) { + Integer id = (Integer) input.get(key); + return id; + } + }; + } + + protected EchoBaseFunctions() { + // helper don't need to be instanciate + } +} Property changes on: trunk/echobase-entities/src/main/java/fr/ifremer/echobase/EchoBaseFunctions.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Modified: trunk/echobase-entities/src/main/resources/i18n/echobase-entities_fr_FR.properties =================================================================== --- trunk/echobase-entities/src/main/resources/i18n/echobase-entities_fr_FR.properties 2011-12-12 13:22:57 UTC (rev 171) +++ trunk/echobase-entities/src/main/resources/i18n/echobase-entities_fr_FR.properties 2011-12-13 13:25:38 UTC (rev 172) @@ -16,7 +16,7 @@ echobase.common.axis= echobase.common.b= echobase.common.baracoudaCode= -echobase.common.bathymetre= +echobase.common.bathymetry= echobase.common.binSizePingAxis= echobase.common.binSizeRangeAxis= echobase.common.binUnitsPingAxis= @@ -43,7 +43,6 @@ echobase.common.dataCentreEmail=Courriel du centre de données echobase.common.dataClass= echobase.common.dataMetadata= -echobase.common.dataName= echobase.common.dataProcessing=dataProcessing echobase.common.dataProtocol= echobase.common.dataQuality=dataQuality @@ -57,7 +56,6 @@ echobase.common.description=Description echobase.common.digitThreshold= echobase.common.distributionStatement= -echobase.common.eIThreshold= echobase.common.echoBaseUser=Utilisateur echobase.common.echoBaseUserDTO= echobase.common.echosounderSoundSpeed= Modified: trunk/echobase-entities/src/main/xmi/echobase.zargo =================================================================== (Binary files differ)

1 0

r171 - trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders
by tchemit＠users.forge.codelutin.com Dec. 12, 2011

Dec. 12, 2011

Author: tchemit Date: 2011-12-12 14:22:57 +0100 (Mon, 12 Dec 2011) New Revision: 171 Url: http://forge.codelutin.com/repositories/revision/echobase/171 Log: clean acoustic data loaders Modified: trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/CellLoader.java trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/DataAcquisitionLoader.java trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/DataProcessingLoader.java trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/EntityLoader.java trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/TransectLoader.java trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/TransitLoader.java trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/VoyageLoader.java Modified: trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/CellLoader.java =================================================================== --- trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/CellLoader.java 2011-12-12 13:13:41 UTC (rev 170) +++ trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/CellLoader.java 2011-12-12 13:22:57 UTC (rev 171) @@ -203,8 +203,8 @@ } protected void loadDatas() throws TopiaException { - List<DataProcessing> dataProcessings = getEntities(DataProcessing.class); - dataProcessingsById = Maps.uniqueIndex(dataProcessings, EchoBaseFunctions.DATA_PROCESSING_ID); + dataProcessingsById = getUniverse(DataProcessing.class, + EchoBaseFunctions.DATA_PROCESSING_ID); DataMetadataDAO dataMetadataDAO = EchoBaseDAOHelper.getDataMetadataDAO(getTransaction()); @@ -225,9 +225,8 @@ //TODO Fix name elementaryTimeMetadata = dataMetadataDAO.findByName("TimeEnd"); - List<DataQuality> dataQualities = getEntities(DataQuality.class); - dataQualitiesById = Maps.uniqueIndex(dataQualities, - EchoBaseFunctions.DATA_QUALITY_ID); + dataQualitiesById = getUniverse( + DataQuality.class, EchoBaseFunctions.DATA_QUALITY_ID); CellTypeDAO cellTypeDAO = EchoBaseDAOHelper.getCellTypeDAO( getTransaction()); @@ -270,7 +269,6 @@ Preconditions.checkNotNull(scenarioByProcessingId); Preconditions.checkNotNull(esduByProcessingId); Preconditions.checkNotNull(trancheDefById); - Preconditions.checkNotNull(trancheDefById); Preconditions.checkNotNull(eiConfigurationByProcessingId); dao = getDAO(); @@ -403,6 +401,8 @@ List<Cell> result = Lists.newArrayList(); + Collection<Map<String, Object>> esduRows = esduByProcessingId.get(processingId); + for (Map<String, Object> scenarioRow : scenarioByProcessingId.get(processingId)) { String rowStrateId = (String) scenarioRow.get("POST_STRATE_SCENARIO"); @@ -425,8 +425,7 @@ log.debug("Treat esdu " + cellStringId); } // keep this line - Collection<Map<String, Object>> esduRows = - esduByProcessingId.get(processingId); + for (Map<String, Object> esduRow : esduRows) { Date rowCellId = (Date) esduRow.get("DHTU_ESDU"); @@ -448,7 +447,7 @@ createEsduCellData(esduCell, cellId, - esduRows, + esduRow, acousticMetadata ); @@ -468,61 +467,52 @@ } private int createEsduCellData(Cell esduCell, Date esduCellId, - Collection<Map<String, Object>> esduRows, + Map<String, Object> esduRow, DataMetadata acousticMetadata ) throws TopiaException { DataQuality dataQuality = esduCell.getDataQuality(); - for (Map<String, Object> row : esduRows) { + Data latitudeData = dataDao.create( + Data.PROPERTY_DATA_NAME, "esduLatitude", + Data.PROPERTY_DATA_METADATA, esduLatitudeMetadata, + Data.PROPERTY_DATA_QUALITY, dataQuality, + Data.PROPERTY_DATA_VALUE, getFloatValue(esduRow, "LATDMC") + ); + esduCell.addData(latitudeData); - // find the correct row - Date rowCellId = (Date) row.get("DHTU_ESDU"); + Data longitudeData = dataDao.create( + Data.PROPERTY_DATA_NAME, "esduLongitude", + Data.PROPERTY_DATA_METADATA, esduLongitudeMetadata, + Data.PROPERTY_DATA_QUALITY, dataQuality, + Data.PROPERTY_DATA_VALUE, getFloatValue(esduRow, "LGDMC") + ); + esduCell.addData(longitudeData); - if (!esduCellId.equals(rowCellId)) { - continue; - } + Data depthData = dataDao.create( + Data.PROPERTY_DATA_NAME, "esduDepth", + Data.PROPERTY_DATA_METADATA, esduDepthMetadata, + Data.PROPERTY_DATA_QUALITY, dataQuality, + Data.PROPERTY_DATA_VALUE, getFloatValue(esduRow, "SONDE") + ); + esduCell.addData(depthData); - Data latitudeData = dataDao.create( - Data.PROPERTY_DATA_NAME, "esduLatitude", - Data.PROPERTY_DATA_METADATA, esduLatitudeMetadata, - Data.PROPERTY_DATA_QUALITY, dataQuality, - Data.PROPERTY_DATA_VALUE, getFloatValue(row, "LATDMC") - ); - esduCell.addData(latitudeData); + Data timeData = dataDao.create( + Data.PROPERTY_DATA_NAME, "esduTime", + Data.PROPERTY_DATA_METADATA, esduTimeMetadata, + Data.PROPERTY_DATA_QUALITY, dataQuality, + Data.PROPERTY_DATA_VALUE, esduCellId.toString() + ); + esduCell.addData(timeData); - Data longitudeData = dataDao.create( - Data.PROPERTY_DATA_NAME, "esduLongitude", - Data.PROPERTY_DATA_METADATA, esduLongitudeMetadata, - Data.PROPERTY_DATA_QUALITY, dataQuality, - Data.PROPERTY_DATA_VALUE, getFloatValue(row, "LGDMC") - ); - esduCell.addData(longitudeData); + Data acousticData = dataDao.create( + Data.PROPERTY_DATA_NAME, "esduAcoustic", + Data.PROPERTY_DATA_METADATA, acousticMetadata, + Data.PROPERTY_DATA_QUALITY, dataQuality, + Data.PROPERTY_DATA_VALUE, getFloatValue(esduRow, "TOTAL") + ); + esduCell.addData(acousticData); - Data depthData = dataDao.create( - Data.PROPERTY_DATA_NAME, "esduDepth", - Data.PROPERTY_DATA_METADATA, esduDepthMetadata, - Data.PROPERTY_DATA_QUALITY, dataQuality, - Data.PROPERTY_DATA_VALUE, getFloatValue(row, "SONDE") - ); - esduCell.addData(depthData); - - Data timeData = dataDao.create( - Data.PROPERTY_DATA_NAME, "esduTime", - Data.PROPERTY_DATA_METADATA, esduTimeMetadata, - Data.PROPERTY_DATA_QUALITY, dataQuality, - Data.PROPERTY_DATA_VALUE, esduCellId.toString() - ); - esduCell.addData(timeData); - - Data acousticData = dataDao.create( - Data.PROPERTY_DATA_NAME, "esduAcoustic", - Data.PROPERTY_DATA_METADATA, acousticMetadata, - Data.PROPERTY_DATA_QUALITY, dataQuality, - Data.PROPERTY_DATA_VALUE, getFloatValue(row, "TOTAL") - ); - esduCell.addData(acousticData); - } return esduCell.sizeData(); } @@ -638,55 +628,6 @@ return value.toString(); } -// private Map<DataProcessing, Map<String, Object>> getDataProcessings(Database db) throws IOException { -// Map<DataProcessing, Map<String, Object>> result = -// Maps.newLinkedHashMap(); -// -// Set<String> allProcessingIds = getAllAvailableProcessingIds(db); -// -// Set<String> noFoundIds = Sets.newHashSet(); -// -// for (Map<String, Object> row : db.getTable("EI_CONFIGURATION")) { -// String processingId = (String) row.get("ID_EI"); -// if (!allProcessingIds.contains(processingId)) { -// //TODO remove this test with new baracouda db -// noFoundIds.add(processingId); -// continue; -// } -// DataProcessing dataProcessing = -// dataProcessingsById.get(processingId.toLowerCase()); -// -// Map<String, Object> postStrateRow = findPostStrateRow( -// db, -// processingId -// ); -// -// result.put(dataProcessing, postStrateRow); -// -// if (postStrateRow == null) { -// if (log.isWarnEnabled()) { -// log.warn("Could not find postStrate with processing Id " + processingId); -// } -// } -// } -// -// for (String noFoundId : noFoundIds) { -// if (log.isWarnEnabled()) { -// log.warn("Could not find data processing data with id " + noFoundId); -// } -// } -// return result; -// } - -// private Set<String> getAllAvailableProcessingIds(Database db) throws IOException { -// Set<String> allProcessingIds = Sets.newHashSet(); -// for (Map<String, Object> row : db.getTable("SCENARIO")) { -// String id = row.get("ID_EI_SCENARIO").toString(); -// allProcessingIds.add(id); -// } -// return allProcessingIds; -// } - private Map<String, Object> findPostStrateRow(Database db, String processingId) throws IOException { Modified: trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/DataAcquisitionLoader.java =================================================================== --- trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/DataAcquisitionLoader.java 2011-12-12 13:13:41 UTC (rev 170) +++ trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/DataAcquisitionLoader.java 2011-12-12 13:22:57 UTC (rev 171) @@ -62,14 +62,11 @@ @Override protected void importBaracoudaEntities(Database db) throws IOException, TopiaException { - List<Voyage> voyages = getEntities(Voyage.class); - Map<String, Voyage> voyagesByName = Maps.uniqueIndex( - voyages, EchoBaseFunctions.VOYAGE_NAME); + Map<String, Voyage> voyagesByName = getUniverse( + Voyage.class, EchoBaseFunctions.VOYAGE_NAME); - List<AcousticInstrument> acousticInstruments = getEntities(AcousticInstrument.class); - Map<String, AcousticInstrument> acousticInstrumentsByName = - Maps.uniqueIndex(acousticInstruments, - EchoBaseFunctions.ACOUSTIC_INSTRUMENT_ID); + Map<String, AcousticInstrument> acousticInstrumentsByName = getUniverse( + AcousticInstrument.class, EchoBaseFunctions.ACOUSTIC_INSTRUMENT_ID); TopiaDAO<DataAcquisition> dao = getDAO(); @@ -81,15 +78,16 @@ for (Map<String, Object> row : db.getTable("EI_CONFIGURATION")) { String voyageName = (String) row.get("CAMPAGNE"); - Voyage voyage = voyagesByName.get(voyageName.toLowerCase()); + Voyage voyage = getFK(voyagesByName, voyageName); Transit transit = voyage.getTransit().iterator().next(); Transect transect = transit.getTransect().iterator().next(); String acousticInstrumentId = row.get("ID_REFSONDEUR").toString(); - AcousticInstrument acousticInstrument = - acousticInstrumentsByName.get(acousticInstrumentId.toLowerCase()); - Map<String, Object> sondeurRow = - EI_SONDEURByVesselName.get(acousticInstrumentId.toLowerCase()); + AcousticInstrument acousticInstrument = getFK( + acousticInstrumentsByName, acousticInstrumentId); + Map<String, Object> sondeurRow = getFK( + EI_SONDEURByVesselName, acousticInstrumentId); + Float transceiverAcquisitionAbsorption = (Float) sondeurRow.get("ALPHA"); Float transceiverAcquisitionPower = (Float) sondeurRow.get("PUISSANCE_MAX"); Modified: trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/DataProcessingLoader.java =================================================================== --- trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/DataProcessingLoader.java 2011-12-12 13:13:41 UTC (rev 170) +++ trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/DataProcessingLoader.java 2011-12-12 13:22:57 UTC (rev 171) @@ -24,7 +24,6 @@ package fr.ifremer.echobase.tools.loaders; import com.google.common.collect.Lists; -import com.google.common.collect.Maps; import com.healthmarketscience.jackcess.Database; import fr.ifremer.echobase.entities.EchoBaseEntityEnum; import fr.ifremer.echobase.entities.data.DataAcquisition; @@ -63,9 +62,8 @@ @Override protected void importBaracoudaEntities(Database db) throws IOException, TopiaException { - List<Voyage> voyages = getEntities(Voyage.class); - Map<String, Voyage> voyagesByName = Maps.uniqueIndex( - voyages, EchoBaseFunctions.VOYAGE_NAME); + Map<String, Voyage> voyagesByName = getUniverse( + Voyage.class, EchoBaseFunctions.VOYAGE_NAME); TopiaDAO<DataProcessing> dao = getDAO(); @@ -73,12 +71,13 @@ for (Map<String, Object> row : db.getTable("EI_CONFIGURATION")) { String voyageName = (String) row.get("CAMPAGNE"); - Voyage voyage = voyagesByName.get(voyageName.toLowerCase()); + Voyage voyage = getFK(voyagesByName,voyageName); Transit transit = voyage.getTransit().iterator().next(); Transect transect = transit.getTransect().iterator().next(); // take the free DataAcquisition - Iterator<DataAcquisition> iterator = transect.getDataAcquisition().iterator(); + Iterator<DataAcquisition> iterator = + transect.getDataAcquisition().iterator(); DataAcquisition dataAcquisition = iterator.next(); while (!dataAcquisition.isDataProcessingEmpty()) { dataAcquisition = iterator.next(); Modified: trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/EntityLoader.java =================================================================== --- trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/EntityLoader.java 2011-12-12 13:13:41 UTC (rev 170) +++ trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/EntityLoader.java 2011-12-12 13:22:57 UTC (rev 171) @@ -24,7 +24,9 @@ package fr.ifremer.echobase.tools.loaders; import com.google.common.base.Charsets; +import com.google.common.base.Function; import com.google.common.base.Preconditions; +import com.google.common.collect.Maps; import com.google.common.collect.Sets; import com.healthmarketscience.jackcess.Database; import fr.ifremer.echobase.EchoBaseTechnicalException; @@ -56,6 +58,8 @@ import java.io.InputStream; import java.io.Writer; import java.text.ParseException; +import java.util.List; +import java.util.Map; import java.util.Set; /** @@ -382,4 +386,30 @@ getDbEditorService().exportDatas(associationMeta, out); } + + protected <E extends TopiaEntity,K> Map<K, E> getUniverse(Class<E> entityType, + Function<E, K> function) { + List<E> entities = getEntities(entityType); + Map<K, E> universe = Maps.uniqueIndex(entities, function); + return universe; + } + + protected <E> E getFK(Map<String, E> universe, String fk) { + return getFK(universe, fk, true, true); + } + + protected <E> E getFK(Map<String, E> universe, String fk, boolean lowerCase, boolean strict) { + E entity = universe.get(lowerCase?fk.toLowerCase():fk); + if (strict) + Preconditions.checkNotNull(entity, "Could not find entity with id " + fk); + else { + if (entity == null) { + if (log.isWarnEnabled()) { + log.warn("Could not find entity with id " + fk); + } + } + } + return entity; + } + } Modified: trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/TransectLoader.java =================================================================== --- trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/TransectLoader.java 2011-12-12 13:13:41 UTC (rev 170) +++ trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/TransectLoader.java 2011-12-12 13:22:57 UTC (rev 171) @@ -24,7 +24,6 @@ package fr.ifremer.echobase.tools.loaders; import com.google.common.collect.Lists; -import com.google.common.collect.Maps; import com.healthmarketscience.jackcess.Database; import com.healthmarketscience.jackcess.Table; import fr.ifremer.echobase.entities.EchoBaseEntityEnum; @@ -62,11 +61,11 @@ @Override protected void importBaracoudaEntities(Database db) throws IOException, TopiaException { - List<Voyage> voyages = getEntities(Voyage.class); - Map<String, Voyage> voyagesByName = Maps.uniqueIndex(voyages, EchoBaseFunctions.VOYAGE_NAME); + Map<String, Voyage> voyagesByName = getUniverse( + Voyage.class, EchoBaseFunctions.VOYAGE_NAME); - List<Vessel> vessels = getEntities(Vessel.class); - Map<String, Vessel> vesselsByName = Maps.uniqueIndex(vessels, EchoBaseFunctions.VESSEL_NAME); + Map<String, Vessel> vesselsByName = getUniverse( + Vessel.class, EchoBaseFunctions.VESSEL_NAME); TopiaDAO<Transect> dao = getDAO(); @@ -75,9 +74,9 @@ for (Map<String, Object> row : table) { String vesselName = (String) row.get("NAVIRE"); - Vessel vessel = vesselsByName.get(vesselName.toLowerCase()); + Vessel vessel = getFK(vesselsByName, vesselName); String voyageName = (String) row.get("CAMPAGNE"); - Voyage voyage = voyagesByName.get(voyageName.toLowerCase()); + Voyage voyage = getFK(voyagesByName, voyageName); Transit transit = voyage.getTransit().iterator().next(); Transect transect = dao.create( Transect.PROPERTY_VESSEL, vessel, Modified: trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/TransitLoader.java =================================================================== --- trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/TransitLoader.java 2011-12-12 13:13:41 UTC (rev 170) +++ trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/TransitLoader.java 2011-12-12 13:22:57 UTC (rev 171) @@ -24,7 +24,6 @@ package fr.ifremer.echobase.tools.loaders; import com.google.common.collect.Lists; -import com.google.common.collect.Maps; import com.healthmarketscience.jackcess.Database; import com.healthmarketscience.jackcess.Table; import fr.ifremer.echobase.entities.EchoBaseEntityEnum; @@ -61,8 +60,8 @@ @Override protected void importBaracoudaEntities(Database db) throws IOException, TopiaException { - List<Voyage> voyages = getEntities(Voyage.class); - Map<String, Voyage> voyagesByName = Maps.uniqueIndex(voyages, EchoBaseFunctions.VOYAGE_NAME); + Map<String, Voyage> voyagesByName = getUniverse( + Voyage.class, EchoBaseFunctions.VOYAGE_NAME); TopiaDAO<Transit> dao = getDAO(); @@ -71,7 +70,7 @@ for (Map<String, Object> row : table) { String voyageName = (String) row.get("CAMPAGNE"); - Voyage voyage = voyagesByName.get(voyageName.toLowerCase()); + Voyage voyage = getFK(voyagesByName, voyageName); Date startTime = voyage.getStartDate(); Date endTime = voyage.getEndDate(); Transit transit = dao.create( Modified: trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/VoyageLoader.java =================================================================== --- trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/VoyageLoader.java 2011-12-12 13:13:41 UTC (rev 170) +++ trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/VoyageLoader.java 2011-12-12 13:22:57 UTC (rev 171) @@ -24,7 +24,6 @@ package fr.ifremer.echobase.tools.loaders; import com.google.common.collect.Lists; -import com.google.common.collect.Maps; import com.healthmarketscience.jackcess.Database; import com.healthmarketscience.jackcess.Table; import fr.ifremer.echobase.entities.data.Voyage; @@ -75,11 +74,12 @@ @Override protected void importBaracoudaEntities(Database db) throws IOException, TopiaException { - List<Mission> missions = getEntities(Mission.class); - Map<String, Mission> missionsByName = Maps.uniqueIndex(missions, EchoBaseFunctions.MISSION_NAME); + Map<String, Mission> missionsByName = getUniverse( + Mission.class, EchoBaseFunctions.MISSION_NAME); - List<AreaOfOperation> areaOfOperations = getEntities(AreaOfOperation.class); - Map<String, AreaOfOperation> areaOfOperationsByName = Maps.uniqueIndex(areaOfOperations, EchoBaseFunctions.AREA_OF_OPERATION_NAME); + Map<String, AreaOfOperation> areaOfOperationsByName = getUniverse( + AreaOfOperation.class, EchoBaseFunctions.AREA_OF_OPERATION_NAME); + TopiaDAO<Voyage> dao = getDAO(); List<Voyage> entities = Lists.newArrayList(); @@ -87,9 +87,10 @@ for (Map<String, Object> row : table) { String voyageName = (String) row.get("CAMPAGNE"); String missionName = (String) row.get("TYPE_CAMPAGNE"); - Mission mission = missionsByName.get(missionName.toLowerCase()); + Mission mission = getFK(missionsByName, missionName); String areaOfOperationName = (String) row.get("ZONE"); - AreaOfOperation areaOfOperation = areaOfOperationsByName.get(areaOfOperationName.toLowerCase()); + AreaOfOperation areaOfOperation = getFK(areaOfOperationsByName, + areaOfOperationName); Date startDate = (Date) row.get("DATE_DEPART"); Date endDate = (Date) row.get("DATE_ARRIVEE"); String startPort = (String) row.get("PORT_DEPART");

1 0

r170 - in trunk/echobase-services/src/main/java/fr/ifremer/echobase/services: . models
by tchemit＠users.forge.codelutin.com Dec. 12, 2011

Dec. 12, 2011

Author: tchemit Date: 2011-12-12 14:13:41 +0100 (Mon, 12 Dec 2011) New Revision: 170 Url: http://forge.codelutin.com/repositories/revision/echobase/170 Log: improve Association entity ie Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbEditorService.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityAssociationCsvModel.java Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbEditorService.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbEditorService.java 2011-12-11 21:40:50 UTC (rev 169) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbEditorService.java 2011-12-12 13:13:41 UTC (rev 170) @@ -48,6 +48,7 @@ import org.nuiton.util.beans.BeanMonitor; import org.nuiton.util.beans.PropertyDiff; import org.nuiton.util.csv.Export; +import org.nuiton.util.csv.ExportModel; import org.nuiton.util.csv.ExportableColumn; import org.nuiton.util.csv.Import; import org.nuiton.util.csv.ImportExportModel; @@ -538,16 +539,16 @@ return model; } - public <E extends TopiaEntity> ImportExportModel<E> buildForExport(AssociationMeta associationMeta) { + public <E extends TopiaEntity> ExportModel<E> buildForExport(AssociationMeta associationMeta) { - ImportExportModel<E> model = EntityAssociationCsvModel.newModel( + ExportModel<E> model = EntityAssociationCsvModel.newExportModel( getConfiguration().getCsvSeparator(), associationMeta ); return model; } - public <E extends TopiaEntity> ImportExportModel<E> buildForExport(MetaFilenameAware meta) { + public <E extends TopiaEntity> ExportModel<E> buildForExport(MetaFilenameAware meta) { if (meta instanceof AssociationMeta) { return buildForExport((AssociationMeta) meta); } @@ -557,7 +558,7 @@ throw new IllegalStateException("Can't do a thing with " + meta); } - public <E extends TopiaEntity> ImportExportModel<E> buildForExport(TableMeta meta) { + public <E extends TopiaEntity> ExportModel<E> buildForExport(TableMeta meta) { EntityCsvModel<E> model = EntityCsvModel.newModel( getConfiguration().getCsvSeparator(), @@ -646,7 +647,7 @@ protected Export<TopiaEntity> prepareExport(TableMeta tableMeta) { List<TopiaEntity> datas = getEntities(tableMeta, null, null, null, null); - ImportExportModel<TopiaEntity> model = buildForExport(tableMeta); + ExportModel<TopiaEntity> model = buildForExport(tableMeta); return Export.newExport(model, datas); } @@ -657,7 +658,7 @@ TableMeta tableMeta = getTableMeta(associationMeta.getSource()); List<TopiaEntity> datas = getEntities(tableMeta, null, null, null, "size(e." + associationMeta.getName() + ") > 0"); - ImportExportModel<TopiaEntity> model = buildForExport(associationMeta); + ExportModel<TopiaEntity> model = buildForExport(associationMeta); return Export.newExport(model, datas); } } Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java 2011-12-11 21:40:50 UTC (rev 169) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java 2011-12-12 13:13:41 UTC (rev 170) @@ -44,11 +44,13 @@ import org.nuiton.util.StringUtil; import org.nuiton.util.csv.Import; import org.nuiton.util.csv.ImportExportModel; +import org.nuiton.util.csv.ImportModel; import java.io.File; import java.io.IOException; import java.io.InputStreamReader; import java.io.Reader; +import java.util.Collection; import java.util.List; import java.util.Map; import java.util.zip.ZipEntry; @@ -194,25 +196,15 @@ AssociationMeta associationMeta = (AssociationMeta) entry; - EchoBaseEntityEnum source = associationMeta.getSource(); - List<TopiaEntity> sourceEntities = getEntities(source); - Map<String, TopiaEntity> sourcesById; - Map<String, TopiaEntity> targetsById; - sourcesById = Maps.uniqueIndex(sourceEntities, CsvModelUtil.TO_TOPIAID); EchoBaseEntityEnum target = associationMeta.getTarget(); - if (source.equals(target)) { - targetsById = sourcesById; - } else { + List<TopiaEntity> targetEntities = getEntities(target); + Map<String, TopiaEntity> targetsById = + Maps.uniqueIndex(targetEntities, CsvModelUtil.TO_TOPIAID); - List<TopiaEntity> targetEntities = getEntities(target); - targetsById = Maps.uniqueIndex(targetEntities, CsvModelUtil.TO_TOPIAID); - } - - ImportExportModel<TopiaEntity> model = - EntityAssociationCsvModel.newModel( + ImportModel<TopiaEntity> model = + EntityAssociationCsvModel.newImportModel( getConfiguration().getCsvSeparator(), associationMeta, - sourcesById, targetsById ); @@ -300,7 +292,7 @@ } protected <E extends TopiaEntity> void importEntityAssociation(AssociationMeta meta, - ImportExportModel<E> csvModel, + ImportModel<E> csvModel, CsvImportResult result, Reader reader) { if (log.isInfoEnabled()) { @@ -308,17 +300,22 @@ } try { - EchoBaseEntityEnum sourceType = meta.getSource(); - EchoBaseEntityEnum targetType = meta.getTarget(); + EchoBaseEntityEnum sourceType = meta.getSource(); +// EchoBaseEntityEnum targetType = meta.getTarget(); - Import<E> importer = Import.newImport(csvModel, reader); + Import<E> importer = Import.newImport(csvModel, reader); TopiaDAO<E> sourceDAO = getDAO(sourceType); - TopiaDAO<TopiaEntity> targetDAO = getDAO(targetType); +// TopiaDAO<TopiaEntity> targetDAO = getDAO(targetType); + List<E> sourceEntities = getEntities(sourceType); + Map<String, E> sourcesById = + Maps.uniqueIndex(sourceEntities, CsvModelUtil.TO_TOPIAID); + try { + for (E row : importer) { + E entityToSave = sourcesById.get(row.getTopiaId()); + Collection<TopiaEntity> assoc = meta.getChilds(row); + meta.setChilds(entityToSave, assoc); +// sourceDAO.update(entityToSave); - try { - for (E row : importer) { - - sourceDAO.update(row); // E entityToSave = sourceDAO.findByTopiaId(row.getTopiaId()); // // // check entity exits ? @@ -338,13 +335,13 @@ // result.incrementsNumberUpdated(); // } // meta.setChilds(entityToSave, associationToSave); - } - } finally { - - importer.close(); } + } finally { - commitTransaction("Could not commit transaction"); + importer.close(); + } + + commitTransaction("Could not commit transaction"); } catch (TopiaException eee) { throw new EchoBaseTechnicalException(eee); } Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java 2011-12-11 21:40:50 UTC (rev 169) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java 2011-12-12 13:13:41 UTC (rev 170) @@ -31,7 +31,6 @@ import org.apache.commons.collections.CollectionUtils; import org.apache.commons.lang3.StringUtils; import org.nuiton.topia.persistence.TopiaEntity; -import org.nuiton.util.ObjectUtil; import org.nuiton.util.StringUtil; import org.nuiton.util.csv.Common; import org.nuiton.util.csv.ValueFormatter; @@ -54,7 +53,7 @@ @Override public String apply(TopiaEntity input) { - return input.getTopiaId().toLowerCase(); + return input.getTopiaId(); } }; @@ -81,6 +80,10 @@ return new ForeignKeyDecoratedValue<E>(entityType, decoratorService, locale); } + public static <E extends TopiaEntity> ValueFormatter<Collection<E>> newAssociationValueFormatter() { + return new AssociationValueParserFormatter<E>(null, null); + } + public static <E extends TopiaEntity> AssociationValueParserFormatter<E> newAssociationValueParserFormatter(Class<E> entityType, Map<String, E> universe) { return new AssociationValueParserFormatter<E>(entityType, universe); @@ -106,8 +109,8 @@ String[] ids = value.split("\\|"); for (String id : ids) { - E association = ObjectUtil.newInstance(entityType); -// E association = universe.get(id); +// E association = ObjectUtil.newInstance(entityType); + E association = universe.get(id); association.setTopiaId(id); result.add(association); } @@ -152,7 +155,7 @@ if (StringUtils.isNotBlank(value)) { // get entity from universe - result = universe.get(value.toLowerCase()); + result = universe.get(value); if (result == null) { Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityAssociationCsvModel.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityAssociationCsvModel.java 2011-12-11 21:40:50 UTC (rev 169) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityAssociationCsvModel.java 2011-12-12 13:13:41 UTC (rev 170) @@ -25,8 +25,10 @@ import fr.ifremer.echobase.entities.meta.AssociationMeta; import org.nuiton.topia.persistence.TopiaEntity; +import org.nuiton.util.csv.ExportModel; import org.nuiton.util.csv.ExportableColumn; import org.nuiton.util.csv.ImportExportModel; +import org.nuiton.util.csv.ImportModel; import org.nuiton.util.csv.ImportableColumn; import org.nuiton.util.csv.ModelBuilder; @@ -48,17 +50,50 @@ protected ModelBuilder<E> modelBuilder; - public static <E extends TopiaEntity> EntityAssociationCsvModel<E> newModel(char separator, - AssociationMeta meta, - Map<String, E> sourcesById, - Map<String, TopiaEntity> targetsById + public static <E extends TopiaEntity> ImportModel<E> newImportModel(char separator, + AssociationMeta meta, + Map<String, TopiaEntity> targetsById ) { EntityAssociationCsvModel<E> model = new EntityAssociationCsvModel<E>( + separator, meta); + + // topiaId <-> topiaId + model.modelBuilder.newColumnForImportExport( + TopiaEntity.TOPIA_ID, + TopiaEntity.TOPIA_ID + ); + + // add association -> target + Class<TopiaEntity> target = + (Class<TopiaEntity>) meta.getTarget().getImplementation(); + + model.modelBuilder.newColumnForImportExport( + meta.getName(), + meta.getName(), + CsvModelUtil.newAssociationValueParserFormatter(target, targetsById) + ); + + return model; + } + + public static <E extends TopiaEntity> ExportModel<E> newExportModel(char separator, + AssociationMeta meta + ) { + EntityAssociationCsvModel<E> model = new EntityAssociationCsvModel<E>( separator, - meta, - sourcesById, - targetsById); + meta); + // topiaId <-> topiaId + model.modelBuilder.newColumnForExport( + TopiaEntity.TOPIA_ID, + TopiaEntity.TOPIA_ID + ); + + model.modelBuilder.newColumnForExport( + meta.getName(), + meta.getName(), + CsvModelUtil.newAssociationValueFormatter() + ); return model; } @@ -88,34 +123,10 @@ return (E) meta.newEntity(); } - protected EntityAssociationCsvModel(char separator, - AssociationMeta meta, - Map<String, E> sourcesById, - Map<String, TopiaEntity> targetsById) { + protected EntityAssociationCsvModel(char separator, AssociationMeta meta) { this.separator = separator; this.meta = meta; modelBuilder = new ModelBuilder<E>(); - - Class<E> source= - (Class<E>) meta.getSource().getContract(); - - // topiaId <-> topiaId - modelBuilder.newColumnForImportExport( - TopiaEntity.TOPIA_ID, - TopiaEntity.TOPIA_ID, - CsvModelUtil.newForeignKeyValue(source, sourcesById) - ); - - // add association -> target - Class<TopiaEntity> target = - (Class<TopiaEntity>) meta.getTarget().getImplementation(); - - modelBuilder.newColumnForImportExport( - meta.getName(), - meta.getName(), - CsvModelUtil.newAssociationValueParserFormatter(target,targetsById) -// CsvModelUtil.newAssociationValueParserFormatter(meta.getTarget().getImplementation()) - ); }

1 0

r169 - trunk/src/doc/reunions
by tchemit＠users.forge.codelutin.com Dec. 11, 2011

Dec. 11, 2011

Author: tchemit Date: 2011-12-11 22:40:50 +0100 (Sun, 11 Dec 2011) New Revision: 169 Url: http://forge.codelutin.com/repositories/revision/echobase/169 Log: fix doc Modified: trunk/src/doc/reunions/reunion-2011-12-05.txt Modified: trunk/src/doc/reunions/reunion-2011-12-05.txt =================================================================== --- trunk/src/doc/reunions/reunion-2011-12-05.txt 2011-12-11 21:40:34 UTC (rev 168) +++ trunk/src/doc/reunions/reunion-2011-12-05.txt 2011-12-11 21:40:50 UTC (rev 169) @@ -93,16 +93,16 @@ On supprime l'entité *AcousticInstrumentInTransect* et on attache directement le *AcousticInstrument* sur le Transect. -Pour les données anciennes on crée une ligne pour chaque Transect. +Pour les données anciennes on crée une ligne pour chaque instrument du Transect. -AcousticInstrument (FK) la référence sur le AcousticInstrument utilisé à partir de *EI_SONDEUR#ID_REFSONDEUR* +AcousticInstrument (FK) la référence sur le AcousticInstrument utilisé à partir de *EI_SONDEUR#ID_REFSONDEUR* TransceiverAcquisitionAbsorption (D) TransceiverAcquisitionAbsorptionDescription (S) TransducerAcquisitionBeamAngleAthwartship (D) TransducerAcquisitionBeamAngleAlongship (D) TransducerAcquisitionPsi (D) TransceiverAcquisitionPower (I) -TransectAcousticInstrumentID (S) inutile on a supprimer cette table d'association +TransectAcousticInstrumentID (S) inutile on a supprimer cette table d'association DataAcquisitionID ID -> inutile car on a déjà une clef technique AcquisitionSoftwareVersion (S) LoggedDataFormat (S) @@ -172,7 +172,7 @@ *SURF* -> pour une cellule de type *RegionSURF* *CLAS* -> pour une cellule de type *RegionCLAS* -On conserve uniquement la donnée suivbante : +On conserve uniquement la donnée suivante : *ID_STRATE* -> *CellName* (S) c'est la clef métier que l'on veut conserver. @@ -221,7 +221,7 @@ injecté dans DataMetadata depuis la table *EI_CONFIGURATION* et qui est lié à la campagne (utilisation du champs EI_CONFIGURATION#ENERG_UNIT_SOURCE). -A noté que pour les données de type *Latitud*, *Longitude* et *depth*, on doit +A noté que pour les données de type *Latitude*, *Longitude* et *depth*, on doit avoir cellMethod = "Start" dans la dataMetadata à utiliser. Cellule de type élémentaire @@ -232,8 +232,8 @@ celluleType nommé *Elementaire* Pour rattacher une cellule de type “élémentaire” à sa cellule ESDU, on utilise -la table *EIT_ENERGIE_TRANCHES* la cellule ESDU = DHTU_ESDU et l'id de la -cellule esdu c'est Id_TRANCHE (mais on ne le conserve pas au final) +la table *EIT_ENERGIES_TRANCHES* la cellule ESDU = DHTU_ESDU et l'id de la +cellule élémentaire c'est ID_TRANCHE (mais on ne le conserve pas au final) Cell ____

1 0

r168 - in trunk/echobase-ui/src/main: java/fr/ifremer/echobase/ui java/fr/ifremer/echobase/ui/actions java/fr/ifremer/echobase/ui/actions/importDb resources resources/config resources/i18n webapp/WEB-INF/includes webapp/WEB-INF/jsp webapp/WEB-INF/jsp/importDb
by tchemit＠users.forge.codelutin.com Dec. 11, 2011

Dec. 11, 2011

Author: tchemit Date: 2011-12-11 22:40:34 +0100 (Sun, 11 Dec 2011) New Revision: 168 Url: http://forge.codelutin.com/repositories/revision/echobase/168 Log: continue db import+ remove legacy import Added: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/Configure.java trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/LaunchImport.java trunk/echobase-ui/src/main/resources/config/struts-importDb.xml trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importDb/ Removed: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importHistorical/ trunk/echobase-ui/src/main/resources/config/struts-importHistorical.xml trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importHistorical/ Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/EchoBaseSession.java trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/EchoBaseTopiaTransactionFilter.java trunk/echobase-ui/src/main/resources/i18n/echobase-ui_fr_FR.properties trunk/echobase-ui/src/main/resources/struts.xml trunk/echobase-ui/src/main/webapp/WEB-INF/includes/header.jsp trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importDb/configure.jsp trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importDb/progress.jsp trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importDb/result.jsp Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/EchoBaseSession.java =================================================================== --- trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/EchoBaseSession.java 2011-12-11 21:39:42 UTC (rev 167) +++ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/EchoBaseSession.java 2011-12-11 21:40:34 UTC (rev 168) @@ -25,7 +25,7 @@ import fr.ifremer.echobase.entities.EchoBaseUser; import fr.ifremer.echobase.services.ImportDataConfiguration; -import fr.ifremer.echobase.services.ImportHistoricalConfiguration; +import fr.ifremer.echobase.services.ImportDbConfiguration; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; @@ -49,8 +49,8 @@ /** Key to keep import data configuration in this session. */ protected static final String PROPERTY_IMPORT_DATA_CONFIGURATION = "importDataConfiguration"; - /** Key to keep historical import configuration in this session. */ - protected static final String PROPERTY_IMPORT_HISTORICAL_CONFIGURATION = "importHistoricalConfiguration"; + /** Key to keep import db configuration in this session. */ + protected static final String PROPERTY_IMPORT_DB_CONFIGURATION = "importDbConfiguration"; /** To store all properties in this session. */ protected Map<String, Object> store; @@ -81,12 +81,12 @@ set(PROPERTY_IMPORT_DATA_CONFIGURATION, configuration); } - public ImportHistoricalConfiguration getImportHistoricalConfiguration() { - return get(PROPERTY_IMPORT_HISTORICAL_CONFIGURATION, ImportHistoricalConfiguration.class); + public ImportDbConfiguration getImportDbConfiguration() { + return get(PROPERTY_IMPORT_DB_CONFIGURATION, ImportDbConfiguration.class); } - public void setImportHistoricalConfiguration(ImportHistoricalConfiguration configuration) { - set(PROPERTY_IMPORT_HISTORICAL_CONFIGURATION, configuration); + public void setImportDbConfiguration(ImportDbConfiguration configuration) { + set(PROPERTY_IMPORT_DB_CONFIGURATION, configuration); } /** Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/EchoBaseTopiaTransactionFilter.java =================================================================== --- trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/EchoBaseTopiaTransactionFilter.java 2011-12-11 21:39:42 UTC (rev 167) +++ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/EchoBaseTopiaTransactionFilter.java 2011-12-11 21:40:34 UTC (rev 168) @@ -52,15 +52,14 @@ applicationContext.getRootContextSupplier(); TopiaContext rootContext = rootContextSupplier.get(); - TopiaContext transaction = null; try { - transaction = rootContext.beginTransaction(); + TopiaContext transaction = rootContext.beginTransaction(); + if (log.isDebugEnabled()) { + log.debug("Starts a new echo transaction " + transaction); + } + return transaction; } catch (TopiaException eee) { throw new TopiaRuntimeException("Could not start transaction", eee); } - if (log.isDebugEnabled()) { - log.debug("Starts a new echo transaction " + transaction); - } - return transaction; } } Added: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/Configure.java =================================================================== --- trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/Configure.java (rev 0) +++ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/Configure.java 2011-12-11 21:40:34 UTC (rev 168) @@ -0,0 +1,100 @@ +/* + * #%L + * EchoBase :: UI + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ +package fr.ifremer.echobase.ui.actions.importDb; + +import fr.ifremer.echobase.InputFile; +import fr.ifremer.echobase.services.ImportDbConfiguration; +import fr.ifremer.echobase.ui.actions.EchoBaseActionSupport; +import org.apache.commons.io.FileUtils; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.nuiton.util.FileUtil; + +import java.io.File; + +/** + * Configure a db import. + * + * @author tchemit <chemit(a)codelutin.com> + * @since 0.2 + */ +public class Configure extends EchoBaseActionSupport { + + private static final long serialVersionUID = 1L; + + /** Logger. */ + private static final Log log = LogFactory.getLog(Configure.class); + + protected ImportDbConfiguration model; + + public ImportDbConfiguration getModel() { + if (model == null) { + model = new ImportDbConfiguration(getLocale()); + } + return model; + } + + public void setInputFileName(String fileName) { + getModel().getInput().setFileName(fileName); + } + + public void setInput(File file) { + getModel().getInput().setFile(file); + } + + public void setInputContentType(String contentType) { + getModel().getInput().setContentType(contentType); + } + + @Override + public String input() throws Exception { + + // always remove configuration from session anytime coming here + getEchoBaseSession().setImportDbConfiguration(null); + return INPUT; + } + + @Override + public String execute() throws Exception { + + ImportDbConfiguration configuration = getModel(); + + File tempDirectory = FileUtils.getTempDirectory(); + File dataDirectory = new File( + tempDirectory, + "echobase-importDb-" + System.currentTimeMillis()); + FileUtil.createDirectoryIfNecessary(dataDirectory); + configuration.setWorkingDirectory(dataDirectory); + if (log.isInfoEnabled()) { + log.info("Temporary directory to use : " + dataDirectory); + } + + InputFile input = configuration.getInput(); + copyFile(input, dataDirectory); + + // store configuration in session + getEchoBaseSession().setImportDbConfiguration(configuration); + return SUCCESS; + } +} Property changes on: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/Configure.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Added: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/LaunchImport.java =================================================================== --- trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/LaunchImport.java (rev 0) +++ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/LaunchImport.java 2011-12-11 21:40:34 UTC (rev 168) @@ -0,0 +1,117 @@ +/* + * #%L + * EchoBase :: UI + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ +package fr.ifremer.echobase.ui.actions.importDb; + +import com.opensymphony.xwork2.ActionContext; +import com.opensymphony.xwork2.Preparable; +import fr.ifremer.echobase.services.CsvImportResult; +import fr.ifremer.echobase.services.DbImportExportService; +import fr.ifremer.echobase.services.ImportDbConfiguration; +import fr.ifremer.echobase.ui.actions.EchoBaseActionSupport; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.nuiton.topia.TopiaContext; +import org.nuiton.topia.framework.TopiaContextImplementor; + +/** + * Start the import db. + * + * @author tchemit <chemit(a)codelutin.com> + * @since 0.2 + */ +public class LaunchImport extends EchoBaseActionSupport implements Preparable { + + private static final long serialVersionUID = 1L; + + /** Logger. */ + private static final Log log = LogFactory.getLog(LaunchImport.class); + + /** Configuration of the historical import. */ + protected ImportDbConfiguration model; + + /** Result of import. */ + protected CsvImportResult importResult; + + /** service to execute the import. */ + protected transient DbImportExportService service; + + protected ActionContext actionContext; + + public ImportDbConfiguration getModel() { + return model; + } + + @Override + public void prepare() throws Exception { + + if (actionContext == null) { + + // keep it since exec and wait then use another thread + actionContext = ActionContext.getContext(); + } else { + + // having the action cotnext here means we laready came here, + // now we need to propagate it + ActionContext.setContext(actionContext); + } + model = getEchoBaseSession().getImportDbConfiguration(); + + service = newService(DbImportExportService.class); + } + + @Override + public String execute() throws Exception { + + // having the action cotnext here means we laready came here, + // now we need to propagate it + ActionContext.setContext(actionContext); + + // we must use a standalone transaction since it will pass through + // in more thna one request + + TopiaContextImplementor tx = (TopiaContextImplementor) serviceContext.getTransaction(); + TopiaContextImplementor rootContext = tx.getRootContext(); + TopiaContext topiaContext = rootContext.beginTransaction(); + serviceContext.setTransaction(topiaContext); + try { + + if (log.isInfoEnabled()) { + log.info("Start imports with file " + + model.getInput().getFileName()); + } + + service.importDb(model, getEchoBaseSession().getEchoBaseUser()); + return SUCCESS; + } finally { + topiaContext.closeContext(); + + // clean configuration + model.destroy(); + + // remove configuration from session + getEchoBaseSession().setImportDbConfiguration(null); + } + } + +} \ No newline at end of file Property changes on: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/importDb/LaunchImport.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Copied: trunk/echobase-ui/src/main/resources/config/struts-importDb.xml (from rev 159, trunk/echobase-ui/src/main/resources/config/struts-importHistorical.xml) =================================================================== --- trunk/echobase-ui/src/main/resources/config/struts-importDb.xml (rev 0) +++ trunk/echobase-ui/src/main/resources/config/struts-importDb.xml 2011-12-11 21:40:34 UTC (rev 168) @@ -0,0 +1,57 @@ +<?xml version="1.0" encoding="UTF-8" ?> + +<!DOCTYPE struts PUBLIC + "-//Apache Software Foundation//DTD Struts Configuration 2.0//EN" + "http://struts.apache.org/dtds/struts-2.1.7.dtd"> + +<struts> + + <package name="importDb" extends="loggued" namespace="/importDb"> + +  + <action name="configureImport" + class="fr.ifremer.echobase.ui.actions.importDb.Configure"> + <interceptor-ref name="paramsPrepareParamsStackLoggued"/> + <result name="input">/WEB-INF/jsp/importDb/configure.jsp</result> + <result type="redirectAction"> + <param name="namespace">/importDb</param> + <param name="actionName">doImport</param> + </result> + </action> + +  + <action name="doImport" + class="fr.ifremer.echobase.ui.actions.importDb.LaunchImport" + method="execute"> + <interceptor-ref name="basicStackLoggued"/> + <interceptor-ref name="execAndWait"/> + <result name="wait">/WEB-INF/jsp/importDb/progress.jsp</result> + <result>/WEB-INF/jsp/importDb/result.jsp</result> + </action> + + </package> + +</struts> + Property changes on: trunk/echobase-ui/src/main/resources/config/struts-importDb.xml ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Deleted: trunk/echobase-ui/src/main/resources/config/struts-importHistorical.xml =================================================================== --- trunk/echobase-ui/src/main/resources/config/struts-importHistorical.xml 2011-12-11 21:39:42 UTC (rev 167) +++ trunk/echobase-ui/src/main/resources/config/struts-importHistorical.xml 2011-12-11 21:40:34 UTC (rev 168) @@ -1,59 +0,0 @@ -<?xml version="1.0" encoding="UTF-8" ?> - - - -<!DOCTYPE struts PUBLIC - "-//Apache Software Foundation//DTD Struts Configuration 2.0//EN" - "http://struts.apache.org/dtds/struts-2.1.7.dtd"> - -<struts> - - <package name="importHistorical" extends="loggued" namespace="/importHistorical"> - -  - <action name="configureImport" - class="fr.ifremer.echobase.ui.actions.importHistorical.Configure"> - <interceptor-ref name="paramsPrepareParamsStackLoggued"/> - <result name="input">/WEB-INF/jsp/importHistorical/configure.jsp</result> - <result type="redirectAction"> - <param name="namespace">/importHistorical</param> - <param name="actionName">doImport</param> - </result> - </action> - -  - <action name="doImport" - class="fr.ifremer.echobase.ui.actions.importHistorical.LaunchImport" - method="execute"> - <interceptor-ref name="basicStackLoggued"/> - <interceptor-ref name="execAndWait"/> - <result name="wait">/WEB-INF/jsp/importHistorical/progress.jsp</result> - <result>/WEB-INF/jsp/importHistorical/result.jsp</result> - </action> - - </package> - -</struts> - Modified: trunk/echobase-ui/src/main/resources/i18n/echobase-ui_fr_FR.properties =================================================================== --- trunk/echobase-ui/src/main/resources/i18n/echobase-ui_fr_FR.properties 2011-12-11 21:39:42 UTC (rev 167) +++ trunk/echobase-ui/src/main/resources/i18n/echobase-ui_fr_FR.properties 2011-12-11 21:40:34 UTC (rev 168) @@ -89,8 +89,8 @@ echobase.legend.importData.configuration.files=Choix des fichiers d'import echobase.legend.importData.configuration.mission=Configurer la mission echobase.legend.importData.configuration.mission.resume=Résumé de la mission à utiliser +echobase.legend.importDb.configuration.files=Choix du fichier d'import de la base complête (fichier zip) echobase.legend.importFileResult=Résultat de l'import %s -echobase.legend.importHistorical.configuration.files=Choix du fichier d'import de données historiques echobase.legend.sqlQuery.configuration=Configuration de la requête echobase.legend.sqlQuery.result=Résultats de la requête echobase.menu.connectToDbInformations=Se connecter à la base @@ -98,7 +98,7 @@ echobase.menu.dbeditor=Modifier les données echobase.menu.export=Exporter des données echobase.menu.importData=Importer des données -echobase.menu.importHistorical=Importer les données historiques +echobase.menu.importDb=Importer une base complête echobase.menu.logs=Visualiser les modifications echobase.menu.users=Gérer les utilisateurs echobase.message.askAdministratorPassword=Demander à un administrateur le mot de passe pour l'utilisateur donné @@ -120,10 +120,10 @@ echobase.title.importData.configuration=Configuration d'un import de données echobase.title.importDataProgress=Import de données en cours echobase.title.importDataResult=Résultats de l'import de données +echobase.title.importDb.configuration=Configuration d'un import de base de données +echobase.title.importDbProgress=Import de données en cours +echobase.title.importDbResult=Résultats de l'import de données echobase.title.importHistorical=Import de données historiques -echobase.title.importHistorical.configuration=Configuration d'un import de données historiques -echobase.title.importHistoricalProgress=Import de données historiques en cours -echobase.title.importHistoricalResult=Résultats de l'import de données historiques echobase.title.importTable=Importer une table echobase.title.login=Connexion echobase.title.modification.detail=Détail de la modification Modified: trunk/echobase-ui/src/main/resources/struts.xml =================================================================== --- trunk/echobase-ui/src/main/resources/struts.xml 2011-12-11 21:39:42 UTC (rev 167) +++ trunk/echobase-ui/src/main/resources/struts.xml 2011-12-11 21:40:34 UTC (rev 168) @@ -132,7 +132,7 @@ <include file="config/struts-dbeditor.xml"/> <include file="config/struts-user.xml"/> <include file="config/struts-importData.xml"/> - <include file="config/struts-importHistorical.xml"/> + <include file="config/struts-importDb.xml"/> <include file="config/struts-export.xml"/> </struts> Modified: trunk/echobase-ui/src/main/webapp/WEB-INF/includes/header.jsp =================================================================== --- trunk/echobase-ui/src/main/webapp/WEB-INF/includes/header.jsp 2011-12-11 21:39:42 UTC (rev 167) +++ trunk/echobase-ui/src/main/webapp/WEB-INF/includes/header.jsp 2011-12-11 21:40:34 UTC (rev 168) @@ -55,8 +55,8 @@ <div class="cleanBoth"> <ul> <li> - <s:a action="configureImport" namespace="/importHistorical" method="input"> - <s:text name="echobase.menu.importHistorical"/></s:a> + <s:a action="configureImport" namespace="/importDb" method="input"> + <s:text name="echobase.menu.importDb"/></s:a> </li> <li> <s:a action="configureMission" namespace="/importData" method="input"> Modified: trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importDb/configure.jsp =================================================================== --- trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importHistorical/configure.jsp 2011-12-08 05:55:37 UTC (rev 159) +++ trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importDb/configure.jsp 2011-12-11 21:40:34 UTC (rev 168) @@ -26,15 +26,15 @@ <title><s:text name="echobase.title.importHistorical"/></title> -<s:form namespace="/importHistorical" method="POST" enctype="multipart/form-data"> +<s:form namespace="/importDb" method="POST" enctype="multipart/form-data"> <fieldset> <legend> - <s:text name="echobase.legend.importHistorical.configuration.files"/> + <s:text name="echobase.legend.importDb.configuration.files"/> </legend> <s:file name="input" required="true" - key="echobase.common.importHistoricalFile"/> + key="echobase.common.importDbFile"/> </fieldset> <br/> Modified: trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importDb/progress.jsp =================================================================== --- trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importHistorical/progress.jsp 2011-12-08 05:55:37 UTC (rev 159) +++ trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importDb/progress.jsp 2011-12-11 21:40:34 UTC (rev 168) @@ -25,10 +25,10 @@ <%@ taglib prefix="s" uri="/struts-tags" %> <%@ taglib prefix="sj" uri="/struts-jquery-tags" %> -<s:url id="url" action="doImport" namespace="/importHistorical"/> +<s:url id="url" action="doImport" namespace="/importDb"/> <meta http-equiv="refresh" content="1;url=${url}"/> -<title><s:text name="echobase.title.importHistoricalProgress"/></title> +<title><s:text name="echobase.title.importDbProgress"/></title> <%-- TODO letellier 20111104 : Add warn icon --%> <p><s:text name="echobase.message.warnImportInProgress"/></p> Modified: trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importDb/result.jsp =================================================================== --- trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importHistorical/result.jsp 2011-12-08 05:55:37 UTC (rev 159) +++ trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/importDb/result.jsp 2011-12-11 21:40:34 UTC (rev 168) @@ -24,7 +24,7 @@ <%@page contentType="text/html" pageEncoding="UTF-8" %> <%@ taglib prefix="s" uri="/struts-tags" %> -<title><s:text name="echobase.title.importHistoricalResult"/></title> +<title><s:text name="echobase.title.importDbResult"/></title> <fieldset> <legend> @@ -37,11 +37,6 @@ <s:label value="%{model.input.fileName}" label="%{getText('echobase.label.importFile')}"/> - <s:label value="%{importResult.numberCreated}" - label="%{getText('echobase.label.numberOfCreatedEntities')}"/> - - <s:label value="%{importResult.numberUpdated}" - label="%{getText('echobase.label.numberOfUpdatedEntities')}"/> </fieldset> <div>TODO</div>

1 0

r167 - in trunk/echobase-services/src/main: java/fr/ifremer/echobase/services java/fr/ifremer/echobase/services/models resources/i18n
by tchemit＠users.forge.codelutin.com Dec. 11, 2011

Dec. 11, 2011

Author: tchemit Date: 2011-12-11 22:39:42 +0100 (Sun, 11 Dec 2011) New Revision: 167 Url: http://forge.codelutin.com/repositories/revision/echobase/167 Log: -remove legacy import code -improve db import -add some little improvments Added: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportDbConfiguration.java Removed: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportHistoricalConfiguration.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportHistoricalService.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/ImportExportModelSupport.java Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/EchoBaseServiceSupport.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityAssociationCsvModel.java trunk/echobase-services/src/main/resources/i18n/echobase-services_fr_FR.properties Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java 2011-12-11 21:37:31 UTC (rev 166) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/DbImportExportService.java 2011-12-11 21:39:42 UTC (rev 167) @@ -33,6 +33,7 @@ import fr.ifremer.echobase.entities.meta.AssociationMeta; import fr.ifremer.echobase.entities.meta.MetaFilenameAware; import fr.ifremer.echobase.entities.meta.TableMeta; +import fr.ifremer.echobase.services.models.CsvModelUtil; import fr.ifremer.echobase.services.models.EntityAssociationCsvModel; import org.apache.commons.io.FileUtils; import org.apache.commons.logging.Log; @@ -48,7 +49,6 @@ import java.io.IOException; import java.io.InputStreamReader; import java.io.Reader; -import java.util.Collection; import java.util.List; import java.util.Map; import java.util.zip.ZipEntry; @@ -71,12 +71,14 @@ /** * Import a complete db from the given zip file. * - * @param file zip file where all csv files are located - * @param user user which perform the import + * @param model model of the db import operation + * @param user user which perform the import * @throws IOException if any io exception while import */ - public void importDb(File file, EchoBaseUser user) throws IOException { + public void importDb(ImportDbConfiguration model, EchoBaseUser user) throws IOException { + File file = model.getInput().getFile(); + ZipFile zipFile = new ZipFile(file); try { MetaFilenameAware[] entries = getEntries(); @@ -89,7 +91,7 @@ // check that all mandatories for (MetaFilenameAware entry : entries) { String filename = entry.getFilename(); - ZipEntry zipEntry = zipFile.getEntry(filename); + ZipEntry zipEntry = zipFile.getEntry("echobase/" + filename); if (zipEntry == null) { missingEntries.add(filename); @@ -110,10 +112,15 @@ // missingEntries + " in import " + file); } + int size = entriestoConsume.size(); + model.setNbSteps(size + 1); + // consume all found entries for (Map.Entry<MetaFilenameAware, ZipEntry> entry : entriestoConsume.entrySet()) { + model.incrementsProgression(); + MetaFilenameAware entryDef = entry.getKey(); ZipEntry value = entry.getValue(); CsvImportResult csvResult = CsvImportResult.newResult( @@ -134,7 +141,7 @@ if (log.isInfoEnabled()) { log.info("Import done with user " + user.getEmail()); } - commitTransaction("Could not commit db import " + file); + commitTransaction("Could not commit db import from file " + file); } finally { zipFile.close(); } @@ -154,11 +161,12 @@ File zipFile = new File(tempDirectory, fileName + ".zip"); DbEditorService service = getDbEditorService(); + File dir = new File(tempDirectory, "echobase"); MetaFilenameAware[] entries = getEntries(); for (MetaFilenameAware entry : entries) { - File entryFile = new File(tempDirectory, entry.getFilename()); + File entryFile = new File(dir, entry.getFilename()); if (entry instanceof AssociationMeta) { AssociationMeta associationMeta = (AssociationMeta) entry; @@ -169,7 +177,7 @@ } } - EchoBaseIOUtil.compressZipFile(zipFile, tempDirectory); + EchoBaseIOUtil.compressZipFile(zipFile, dir); return zipFile; } @@ -186,10 +194,26 @@ AssociationMeta associationMeta = (AssociationMeta) entry; + EchoBaseEntityEnum source = associationMeta.getSource(); + List<TopiaEntity> sourceEntities = getEntities(source); + Map<String, TopiaEntity> sourcesById; + Map<String, TopiaEntity> targetsById; + sourcesById = Maps.uniqueIndex(sourceEntities, CsvModelUtil.TO_TOPIAID); + EchoBaseEntityEnum target = associationMeta.getTarget(); + if (source.equals(target)) { + targetsById = sourcesById; + } else { + + List<TopiaEntity> targetEntities = getEntities(target); + targetsById = Maps.uniqueIndex(targetEntities, CsvModelUtil.TO_TOPIAID); + } + ImportExportModel<TopiaEntity> model = EntityAssociationCsvModel.newModel( getConfiguration().getCsvSeparator(), - associationMeta + associationMeta, + sourcesById, + targetsById ); importEntityAssociation(associationMeta, model, csvResult, reader); @@ -200,7 +224,8 @@ TableMeta tableMeta = (TableMeta) entry; - ImportExportModel<TopiaEntity> model = service.buildForImport(tableMeta); + ImportExportModel<TopiaEntity> model = + service.buildForImport(tableMeta); importEntities(tableMeta, model, csvResult, reader); } @@ -208,27 +233,31 @@ public MetaFilenameAware[] getEntries() { - List<MetaFilenameAware> result = Lists.newArrayList(); + List<MetaFilenameAware> entities = Lists.newArrayList(); + List<MetaFilenameAware> associations = Lists.newArrayList(); - addEntries(result, EntitiesUtil.getReferenceTypesForCopy()); - addEntries(result, EntitiesUtil.getDataTypesforCopy()); - return result.toArray(new MetaFilenameAware[result.size()]); + addEntries(entities, associations, EntitiesUtil.getReferenceTypesForCopy()); + addEntries(entities, associations, EntitiesUtil.getDataTypesforCopy()); + entities.addAll(associations); + return entities.toArray(new MetaFilenameAware[entities.size()]); } protected TableMeta getTableMeta(EchoBaseEntityEnum tableName) { return getDbEditorService().getTableMeta(tableName); } - protected void addEntries(List<MetaFilenameAware> result, + protected void addEntries(List<MetaFilenameAware> entities, + List<MetaFilenameAware> associations, EchoBaseEntityEnum[] types) { for (EchoBaseEntityEnum type : types) { TableMeta tableMeta = getTableMeta(type); - result.add(tableMeta); + entities.add(tableMeta); - for (AssociationMeta associationMeta : tableMeta.getAssociations()) { - result.add(associationMeta); - } +// for (AssociationMeta associationMeta : tableMeta.getAssociations()) { +// associations.add(associationMeta); +// } + associations.addAll(tableMeta.getAssociations()); } } @@ -244,9 +273,11 @@ CsvImportResult result, Reader reader) { + if (log.isInfoEnabled()) { + log.info("Will import " + meta); + } Import<TopiaEntity> importer = Import.newImport(csvModel, reader); - try { TopiaDAO<TopiaEntity> dao = getDAO(meta.getSource()); @@ -272,7 +303,9 @@ ImportExportModel<E> csvModel, CsvImportResult result, Reader reader) { - + if (log.isInfoEnabled()) { + log.info("Will import " + meta); + } try { EchoBaseEntityEnum sourceType = meta.getSource(); @@ -285,25 +318,26 @@ try { for (E row : importer) { - E entityToSave = sourceDAO.findByTopiaId(row.getTopiaId()); - - // check entity exits ? - if (entityToSave == null) { - throw new EchoBaseTechnicalException( - "Could not find entity with id " + row.getTopiaId()); - } - Collection<TopiaEntity> assoc = meta.getChilds(row); - Collection<TopiaEntity> associationToSave = Lists.newArrayList(); - for (TopiaEntity topiaEntity : assoc) { - TopiaEntity byTopiaId = targetDAO.findByTopiaId(topiaEntity.getTopiaId()); - if (byTopiaId == null) { - throw new EchoBaseTechnicalException( - "Could not find entity with id " + topiaEntity.getTopiaId()); - } - associationToSave.add(byTopiaId); - result.incrementsNumberUpdated(); - } - meta.setChilds(entityToSave, associationToSave); + sourceDAO.update(row); +// E entityToSave = sourceDAO.findByTopiaId(row.getTopiaId()); +// +// // check entity exits ? +// if (entityToSave == null) { +// throw new EchoBaseTechnicalException( +// "Could not find entity with id " + row.getTopiaId()); +// } +// Collection<TopiaEntity> assoc = meta.getChilds(row); +// Collection<TopiaEntity> associationToSave = Lists.newArrayList(); +// for (TopiaEntity topiaEntity : assoc) { +// TopiaEntity byTopiaId = targetDAO.findByTopiaId(topiaEntity.getTopiaId()); +// if (byTopiaId == null) { +// throw new EchoBaseTechnicalException( +// "Could not find entity with id " + topiaEntity.getTopiaId()); +// } +// associationToSave.add(byTopiaId); +// result.incrementsNumberUpdated(); +// } +// meta.setChilds(entityToSave, associationToSave); } } finally { @@ -315,5 +349,4 @@ throw new EchoBaseTechnicalException(eee); } } - } Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/EchoBaseServiceSupport.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/EchoBaseServiceSupport.java 2011-12-11 21:37:31 UTC (rev 166) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/EchoBaseServiceSupport.java 2011-12-11 21:39:42 UTC (rev 167) @@ -53,6 +53,10 @@ this.serviceContext = serviceContext; } + public <E extends TopiaEntity> List<E> getEntities(EchoBaseEntityEnum entityType) { + Class<E> contract = (Class<E>) entityType.getContract(); + return getEntities(contract); + } public <E extends TopiaEntity> List<E> getEntities(Class<E> entityType) { Preconditions.checkNotNull(entityType); try { Added: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportDbConfiguration.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportDbConfiguration.java (rev 0) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportDbConfiguration.java 2011-12-11 21:39:42 UTC (rev 167) @@ -0,0 +1,101 @@ +/* + * #%L + * EchoBase :: Services + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ +package fr.ifremer.echobase.services; + +import fr.ifremer.echobase.InputFile; +import org.apache.commons.io.FileUtils; + +import java.io.File; +import java.io.IOException; +import java.io.Serializable; +import java.util.Locale; + +import static org.nuiton.i18n.I18n.l_; + +/** + * Configuration of a import db operation. + * + * @author tchemit <chemit(a)codelutin.com> + * @since 0.2 + */ +public class ImportDbConfiguration implements Serializable { + + private static final long serialVersionUID = 1L; + + protected File workingDirectory; + + protected InputFile input; + + protected int nbSteps; + + private float stepIncrement; + + protected float progression; + + public ImportDbConfiguration(Locale locale) { + input = InputFile.newFile( + l_(locale, "echobase.common.importDbFile")); + } + + public File getWorkingDirectory() { + return workingDirectory; + } + + public InputFile getInput() { + return input; + } + + public int getNbSteps() { + return nbSteps; + } + + public float getProgression() { + return progression; + } + + public void setWorkingDirectory(File workingDirectory) { + this.workingDirectory = workingDirectory; + } + + public void setProgression(float progression) { + this.progression = progression; + } + + public void incrementsProgression() { + setProgression(progression + stepIncrement); + } + + public void setNbSteps(int nbSteps) { + + this.nbSteps = nbSteps; + stepIncrement = 100f / nbSteps; + } + + public void destroy() throws IOException { + if (workingDirectory != null) { + FileUtils.deleteDirectory(workingDirectory); + } + } + +} Property changes on: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportDbConfiguration.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Deleted: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportHistoricalConfiguration.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportHistoricalConfiguration.java 2011-12-11 21:37:31 UTC (rev 166) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportHistoricalConfiguration.java 2011-12-11 21:39:42 UTC (rev 167) @@ -1,102 +0,0 @@ -/* - * #%L - * EchoBase :: Services - * - * $Id$ - * $HeadURL$ - * %% - * Copyright (C) 2011 Ifremer, Codelutin - * %% - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU Affero General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see <http://www.gnu.org/licenses/>. - * #L% - */ -package fr.ifremer.echobase.services; - -import fr.ifremer.echobase.InputFile; -import org.apache.commons.io.FileUtils; - -import java.io.File; -import java.io.IOException; -import java.io.Serializable; -import java.util.Locale; - -import static org.nuiton.i18n.I18n.l_; - -/** - * Configuration of an historical import. - * - * @author tchemit <chemit(a)codelutin.com> - * @since 0.2 - */ -public class ImportHistoricalConfiguration implements Serializable { - - private static final long serialVersionUID = 1L; - - protected File workingDirectory; - - - protected InputFile input; - - protected int nbSteps; - - private float stepIncrement; - - protected float progression; - - public ImportHistoricalConfiguration(Locale locale) { - input = InputFile.newFile( - l_(locale, "echobase.common.importHistoricalFile")); - } - - public File getWorkingDirectory() { - return workingDirectory; - } - - public InputFile getInput() { - return input; - } - - public int getNbSteps() { - return nbSteps; - } - - public float getProgression() { - return progression; - } - - public void setWorkingDirectory(File workingDirectory) { - this.workingDirectory = workingDirectory; - } - - public void setProgression(float progression) { - this.progression = progression; - } - - public void incrementsProgression() { - setProgression(progression + stepIncrement); - } - - public void computeSteps() { - nbSteps = 0; - - stepIncrement = 100f / nbSteps; - } - - public void destroy() throws IOException { - if (workingDirectory != null) { - FileUtils.deleteDirectory(workingDirectory); - } - } - -} \ No newline at end of file Deleted: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportHistoricalService.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportHistoricalService.java 2011-12-11 21:37:31 UTC (rev 166) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/ImportHistoricalService.java 2011-12-11 21:39:42 UTC (rev 167) @@ -1,94 +0,0 @@ -/* - * #%L - * EchoBase :: Services - * - * $Id$ - * $HeadURL$ - * %% - * Copyright (C) 2011 Ifremer, Codelutin - * %% - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU Affero General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see <http://www.gnu.org/licenses/>. - * #L% - */ -package fr.ifremer.echobase.services; - -import fr.ifremer.echobase.EchoBaseTechnicalException; -import fr.ifremer.echobase.InputFile; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; - -import java.io.File; - -/** - * To import historical datas (from access database). - * - * @author tchemit <chemit(a)codelutin.com> - * @since 0.2 - */ -public class ImportHistoricalService extends EchoBaseServiceSupport { - - /** Logger. */ - private static final Log log = - LogFactory.getLog(ImportHistoricalService.class); - - public CsvImportResult startImport(ImportHistoricalConfiguration configuration) { - - CsvImportResult result; - - InputFile input = configuration.getInput(); - - if (log.isInfoEnabled()) { - log.info("Starts historical import with file " + - input.getFileName()); - } - - try { - - result = doImport(input); - - } finally { - deleteFile(input); - } - return result; - } - - protected CsvImportResult doImport(InputFile input) { - - CsvImportResult result = CsvImportResult.newResult(null, input.getFileName(), false); - prepareImport(); - return result; - } - - protected void deleteFile(InputFile input) { - File file = input.getFile(); - - if (log.isInfoEnabled()) { - log.info("Will delete import file " + file); - } - boolean wasDel = file.delete(); - if (!wasDel) { - throw new EchoBaseTechnicalException("Could not delete file " + file); - } - } - - - protected void prepareImport() { - try { - Thread.sleep(2000); - } catch (InterruptedException e) { - // ignore - } - } - -} Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java 2011-12-11 21:37:31 UTC (rev 166) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/CsvModelUtil.java 2011-12-11 21:39:42 UTC (rev 167) @@ -50,21 +50,30 @@ */ public class CsvModelUtil extends Common { + public static final Function<TopiaEntity, String> TO_TOPIAID = new Function<TopiaEntity, String>() { + + @Override + public String apply(TopiaEntity input) { + return input.getTopiaId().toLowerCase(); + } + }; + public static <E extends TopiaEntity> ForeignKeyValue<E> newForeignKeyValue(Class<E> type) { return new ForeignKeyValue<E>(type); } public static <E extends TopiaEntity> ForeignKeyValue<E> newForeignKeyValue(Class<E> type, Collection<E> entitites) { - return new ForeignKeyValue<E>(type, entitites, new Function<E, String>() { - @Override - public String apply(E input) { - return input.getTopiaId().toLowerCase(); - } - }); + Map<String, E> universe = Maps.uniqueIndex(entitites, TO_TOPIAID); + return newForeignKeyValue(type, universe); } + public static <E extends TopiaEntity> ForeignKeyValue<E> newForeignKeyValue(Class<E> type, Map<String, E> universe) { + return new ForeignKeyValue<E>(type, universe); + } + public static <E extends TopiaEntity> ForeignKeyValue<E> newForeignKeyValue(Class<E> type, Collection<E> entitites, Function<E, String> transform) { - return new ForeignKeyValue<E>(type, entitites, transform); + Map<String, E> universe = Maps.uniqueIndex(entitites, transform); + return new ForeignKeyValue<E>(type, universe); } public static <E extends TopiaEntity> ForeignKeyDecoratedValue<E> newForeignKeyDecoratedValue(Class<E> entityType, @@ -72,16 +81,22 @@ return new ForeignKeyDecoratedValue<E>(entityType, decoratorService, locale); } - public static <E extends TopiaEntity> AssociationValueParserFormatter<E> newAssociationValueParserFormatter(Class<E> type) { - return new AssociationValueParserFormatter<E>(type); + public static <E extends TopiaEntity> AssociationValueParserFormatter<E> newAssociationValueParserFormatter(Class<E> entityType, + Map<String, E> universe) { + return new AssociationValueParserFormatter<E>(entityType, universe); } public static class AssociationValueParserFormatter<E extends TopiaEntity> implements ValueParserFormatter<Collection<E>> { protected final Class<E> entityType; - public AssociationValueParserFormatter(Class<E> entityType) { + protected final Map<String, E> universe; + + public AssociationValueParserFormatter( + Class<E> entityType, + Map<String, E> universe) { this.entityType = entityType; + this.universe = universe; } @Override @@ -92,6 +107,7 @@ String[] ids = value.split("\\|"); for (String id : ids) { E association = ObjectUtil.newInstance(entityType); +// E association = universe.get(id); association.setTopiaId(id); result.add(association); } @@ -120,10 +136,9 @@ protected final Map<String, E> universe; public ForeignKeyValue(Class<E> entityType, - Collection<E> entitites, - Function<E, String> transform) { + Map<String, E> universe) { this.entityType = entityType; - universe = Maps.uniqueIndex(entitites, transform); + this.universe = universe; } public ForeignKeyValue(Class<E> entityType) { Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityAssociationCsvModel.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityAssociationCsvModel.java 2011-12-11 21:37:31 UTC (rev 166) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/EntityAssociationCsvModel.java 2011-12-11 21:39:42 UTC (rev 167) @@ -32,6 +32,7 @@ import java.util.Collection; import java.util.List; +import java.util.Map; /** * A model to import / export associations of entities into csv files. @@ -48,8 +49,17 @@ protected ModelBuilder<E> modelBuilder; public static <E extends TopiaEntity> EntityAssociationCsvModel<E> newModel(char separator, - AssociationMeta meta) { - return new EntityAssociationCsvModel<E>(separator, meta); + AssociationMeta meta, + Map<String, E> sourcesById, + Map<String, TopiaEntity> targetsById + ) { + EntityAssociationCsvModel<E> model = new EntityAssociationCsvModel<E>( + separator, + meta, + sourcesById, + targetsById); + + return model; } @Override @@ -79,22 +89,32 @@ } protected EntityAssociationCsvModel(char separator, - AssociationMeta meta) { + AssociationMeta meta, + Map<String, E> sourcesById, + Map<String, TopiaEntity> targetsById) { this.separator = separator; this.meta = meta; modelBuilder = new ModelBuilder<E>(); + Class<E> source= + (Class<E>) meta.getSource().getContract(); + // topiaId <-> topiaId modelBuilder.newColumnForImportExport( TopiaEntity.TOPIA_ID, - TopiaEntity.TOPIA_ID + TopiaEntity.TOPIA_ID, + CsvModelUtil.newForeignKeyValue(source, sourcesById) ); // add association -> target + Class<TopiaEntity> target = + (Class<TopiaEntity>) meta.getTarget().getImplementation(); + modelBuilder.newColumnForImportExport( meta.getName(), meta.getName(), - CsvModelUtil.newAssociationValueParserFormatter(meta.getTarget().getImplementation()) + CsvModelUtil.newAssociationValueParserFormatter(target,targetsById) +// CsvModelUtil.newAssociationValueParserFormatter(meta.getTarget().getImplementation()) ); } Deleted: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/ImportExportModelSupport.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/ImportExportModelSupport.java 2011-12-11 21:37:31 UTC (rev 166) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/models/ImportExportModelSupport.java 2011-12-11 21:39:42 UTC (rev 167) @@ -1,71 +0,0 @@ -/* - * #%L - * EchoBase :: Services - * - * $Id$ - * $HeadURL$ - * %% - * Copyright (C) 2011 Ifremer, Codelutin - * %% - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU Affero General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see <http://www.gnu.org/licenses/>. - * #L% - */ -package fr.ifremer.echobase.services.models; - -import org.nuiton.util.csv.ExportableColumn; -import org.nuiton.util.csv.ImportExportModel; -import org.nuiton.util.csv.ImportableColumn; -import org.nuiton.util.csv.ModelBuilder; - -import java.util.Collection; -import java.util.List; - -/** - * TODO - * - * @author tchemit <chemit(a)codelutin.com> - * @since TODO - */ -public abstract class ImportExportModelSupport<E> implements ImportExportModel<E> { - - protected char separator; - - protected ModelBuilder<E> modelBuilder; - - protected ImportExportModelSupport(char separator) { - this.separator = separator; - modelBuilder = new ModelBuilder<E>(); - } - - @Override - public char getSeparator() { - return separator; - } - - @Override - public void pushCsvHeaderNames(List<String> headerNames) { - } - - @Override - public Collection<ExportableColumn<E, Object>> getColumnsForExport() { - return (Collection) - modelBuilder.getColumnsForExport(); - } - - @Override - public Collection<ImportableColumn<E, Object>> getColumnsForImport() { - return (Collection) - modelBuilder.getColumnsForImport(); - } -} Modified: trunk/echobase-services/src/main/resources/i18n/echobase-services_fr_FR.properties =================================================================== --- trunk/echobase-services/src/main/resources/i18n/echobase-services_fr_FR.properties 2011-12-11 21:37:31 UTC (rev 166) +++ trunk/echobase-services/src/main/resources/i18n/echobase-services_fr_FR.properties 2011-12-11 21:39:42 UTC (rev 167) @@ -3,6 +3,6 @@ echobase.common.eventsImport=Import Evènements echobase.common.importDataMode.acoustic=Import Acoustic echobase.common.importDataMode.all=Import globale -echobase.common.importHistoricalFile=Fichier MS-ACCESS de données historiques +echobase.common.importDbFile=File d'import (zip) echobase.common.lectureAgeGenImport=Import Lecture Agen Gen echobase.common.typeEchoSpeciesImport=Import Type EchoSpecies

1 0

r166 - trunk/echobase-entities/src/main/java/fr/ifremer/echobase/entities
by tchemit＠users.forge.codelutin.com Dec. 11, 2011

Dec. 11, 2011

Author: tchemit Date: 2011-12-11 22:37:31 +0100 (Sun, 11 Dec 2011) New Revision: 166 Url: http://forge.codelutin.com/repositories/revision/echobase/166 Log: add references entities used Modified: trunk/echobase-entities/src/main/java/fr/ifremer/echobase/entities/EntitiesUtil.java Modified: trunk/echobase-entities/src/main/java/fr/ifremer/echobase/entities/EntitiesUtil.java =================================================================== --- trunk/echobase-entities/src/main/java/fr/ifremer/echobase/entities/EntitiesUtil.java 2011-12-11 19:30:13 UTC (rev 165) +++ trunk/echobase-entities/src/main/java/fr/ifremer/echobase/entities/EntitiesUtil.java 2011-12-11 21:37:31 UTC (rev 166) @@ -146,6 +146,7 @@ public static EchoBaseEntityEnum[] getReferenceTypesForCopy() { List<EchoBaseEntityEnum> result = Lists.newLinkedList(); + result.add(EchoBaseEntityEnum.Localisation); result.add(EchoBaseEntityEnum.SexeCategory); result.add(EchoBaseEntityEnum.AgeCategory); result.add(EchoBaseEntityEnum.SizeCategory); @@ -164,7 +165,9 @@ result.add(EchoBaseEntityEnum.ReferenceDatumType); result.add(EchoBaseEntityEnum.ReferenceDatum); result.add(EchoBaseEntityEnum.DataType); + result.add(EchoBaseEntityEnum.DataQuality); result.add(EchoBaseEntityEnum.CellMethod); + result.add(EchoBaseEntityEnum.CellType); result.add(EchoBaseEntityEnum.DataMetadata); return result.toArray(new EchoBaseEntityEnum[result.size()]); }

1 0

r165 - in trunk/echobase-tools: . src/main/resources/csv/baracouda src/test/java/fr/ifremer/echobase/tools src/test/java/fr/ifremer/echobase/tools/loaders
by tchemit＠users.forge.codelutin.com Dec. 11, 2011

Dec. 11, 2011

Author: tchemit Date: 2011-12-11 20:30:13 +0100 (Sun, 11 Dec 2011) New Revision: 165 Url: http://forge.codelutin.com/repositories/revision/echobase/165 Log: change some tests to IT launchable via maven profiles, svn properties Added: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbZipCreatorIT.java trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseMSAccessToCsvIT.java trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellByDataBadProcessingLoaderTest.java trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellByDataProcessingLoaderTest.java Removed: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbCreatorTest.java trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellLoaderByDataBadProcessingTest.java trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellLoaderByDataProcessingLoaderTest.java Modified: trunk/echobase-tools/pom.xml trunk/echobase-tools/src/main/resources/csv/baracouda/Data.csv Modified: trunk/echobase-tools/pom.xml =================================================================== --- trunk/echobase-tools/pom.xml 2011-12-11 18:56:22 UTC (rev 164) +++ trunk/echobase-tools/pom.xml 2011-12-11 19:30:13 UTC (rev 165) @@ -154,6 +154,7 @@ </build> <profiles> + <profile> <id>run-loaders</id> <activation> @@ -176,6 +177,42 @@ </plugins> </build> </profile> + + <profile> + <id>msaccess-to-csv</id> + + <build> + <defaultGoal>test</defaultGoal> + <plugins> + <plugin> + <artifactId>maven-surefire-plugin</artifactId> + <configuration> + <includes> + <include>**/*EchoBaseMSAccessToCsvIT.java</include> + </includes> + </configuration> + </plugin> + </plugins> + </build> + </profile> + + <profile> + <id>db-to-zip</id> + + <build> + <defaultGoal>test</defaultGoal> + <plugins> + <plugin> + <artifactId>maven-surefire-plugin</artifactId> + <configuration> + <includes> + <include>**/*EchoBaseDbZipCreatorIT.java</include> + </includes> + </configuration> + </plugin> + </plugins> + </build> + </profile> </profiles> </project> Property changes on: trunk/echobase-tools/src/main/resources/csv/baracouda/Data.csv ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Deleted: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbCreatorTest.java =================================================================== --- trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbCreatorTest.java 2011-12-11 18:56:22 UTC (rev 164) +++ trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbCreatorTest.java 2011-12-11 19:30:13 UTC (rev 165) @@ -1,118 +0,0 @@ -/* - * #%L - * EchoBase :: Tools - * - * $Id$ - * $HeadURL$ - * %% - * Copyright (C) 2011 Ifremer, Codelutin - * %% - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU Affero General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see <http://www.gnu.org/licenses/>. - * #L% - */ -package fr.ifremer.echobase.tools; - -import fr.ifremer.echobase.tools.loaders.AbstractLoaderTest; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.junit.Assert; -import org.junit.Assume; -import org.junit.Before; -import org.junit.BeforeClass; -import org.junit.Test; -import org.nuiton.util.FileUtil; - -import java.io.File; - -/** - * To test {@link EchoBaseDbCreator}. - * - * @author tchemit <chemit(a)codelutin.com> - * @since 0.2 - */ -public class EchoBaseDbCreatorTest extends AbstractToolTest { - - /** Logger. */ - private static final Log log = - LogFactory.getLog(EchoBaseDbCreatorTest.class); - - protected static File msAccessFile; - - protected File exportdir; - - @BeforeClass - public static void beforeClass() { - AbstractLoaderTest.beforeClass(); - - msAccessFile = FileUtil.getFileFromPaths(TestHelper.getBasedir(), - "src", - "test", - "private", - "baracouda.mdb" - ); - if (!msAccessFile.exists()) { - if (log.isWarnEnabled()) { - log.warn("Could not find " + msAccessFile + - ", the test will be skip."); - } - } - Assume.assumeTrue(msAccessFile.exists()); - } - - @Before - public void setUp() throws Exception { - - super.setUp(); - - exportdir = new File(testdir, "export"); - - // exportdir should not exist - Assert.assertFalse(exportdir.exists()); - - // creates it - FileUtil.createDirectoryIfNecessary(exportdir); - } - - @Test - public void createEchoBaseDb() throws Exception { - - File dir = new File(exportdir, "echobase"); - - if (log.isInfoEnabled()) { - log.info("Generate echobase csv files in " + dir); - } - - File distributionFile = new File(exportdir, "echobase.zip"); - Assert.assertFalse(distributionFile.exists()); - - File basedir = TestHelper.getBasedir(); - File sourceDir = FileUtil.getFileFromFQN(basedir, "src.main.resources.csv"); - new EchoBaseDbCreator().createEchoBaseDb(sourceDir, dir, distributionFile); - - Assert.assertTrue(distributionFile.exists()); - } - - @Test - public void exportMSAccessToCsv() throws Exception { - - File dir = new File(exportdir, "msaccessToCsv"); - - if (log.isInfoEnabled()) { - log.info("Generate msaccess files to csv in " + dir); - } - - new EchoBaseDbCreator().exportMSAccessToCsv(msAccessFile, dir); - } - -} Copied: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbZipCreatorIT.java (from rev 163, trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbCreatorTest.java) =================================================================== --- trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbZipCreatorIT.java (rev 0) +++ trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbZipCreatorIT.java 2011-12-11 19:30:13 UTC (rev 165) @@ -0,0 +1,65 @@ +/* + * #%L + * EchoBase :: Tools + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ +package fr.ifremer.echobase.tools; + +import org.junit.Assert; +import org.junit.Test; +import org.nuiton.util.FileUtil; + +import java.io.File; + +/** + * To test {@link EchoBaseDbCreator}. + * + * @author tchemit <chemit(a)codelutin.com> + * @since 0.2 + */ +public class EchoBaseDbZipCreatorIT extends AbstractToolTest { + + protected File exportdir; + + @Test + public void createEchoBaseDb() throws Exception { + + exportdir = new File(testdir, "export"); + + // exportdir should not exist + Assert.assertFalse(exportdir.exists()); + + // creates it + FileUtil.createDirectoryIfNecessary(exportdir); + + File dir = new File(exportdir, "echobase"); + + File distributionFile = new File(exportdir, "echobase.zip"); + Assert.assertFalse(distributionFile.exists()); + + File basedir = TestHelper.getBasedir(); + File sourceDir = FileUtil.getFileFromFQN(basedir, "src.main.resources.csv"); + new EchoBaseDbCreator().createEchoBaseDb(sourceDir, dir, distributionFile); + + Assert.assertTrue(distributionFile.exists()); + } + +} Property changes on: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbZipCreatorIT.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Copied: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseMSAccessToCsvIT.java (from rev 163, trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseDbCreatorTest.java) =================================================================== --- trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseMSAccessToCsvIT.java (rev 0) +++ trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/EchoBaseMSAccessToCsvIT.java 2011-12-11 19:30:13 UTC (rev 165) @@ -0,0 +1,85 @@ +/* + * #%L + * EchoBase :: Tools + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ +package fr.ifremer.echobase.tools; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Assert; +import org.junit.Assume; +import org.junit.Test; +import org.nuiton.util.FileUtil; + +import java.io.File; + +/** + * To test {@link EchoBaseDbCreator}. + * + * @author tchemit <chemit(a)codelutin.com> + * @since 0.2 + */ +public class EchoBaseMSAccessToCsvIT extends AbstractToolTest { + + /** Logger. */ + private static final Log log = + LogFactory.getLog(EchoBaseMSAccessToCsvIT.class); + + protected File msAccessFile; + + protected File exportdir; + + @Test + public void exportMSAccessToCsv() throws Exception { + + msAccessFile = FileUtil.getFileFromPaths(TestHelper.getBasedir(), + "src", + "test", + "private", + "baracouda.mdb" + ); + if (!msAccessFile.exists()) { + if (log.isWarnEnabled()) { + log.warn("Could not find " + msAccessFile + + ", the test will be skip."); + } + } + Assume.assumeTrue(msAccessFile.exists()); + + exportdir = new File(testdir, "export"); + + // exportdir should not exist + Assert.assertFalse(exportdir.exists()); + + // creates it + FileUtil.createDirectoryIfNecessary(exportdir); + + File dir = new File(exportdir, "msaccessToCsv"); + + if (log.isInfoEnabled()) { + log.info("Generate msaccess files to csv in " + dir); + } + + new EchoBaseDbCreator().exportMSAccessToCsv(msAccessFile, dir); + } + +} Copied: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellByDataBadProcessingLoaderTest.java (from rev 163, trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellLoaderByDataBadProcessingTest.java) =================================================================== --- trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellByDataBadProcessingLoaderTest.java (rev 0) +++ trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellByDataBadProcessingLoaderTest.java 2011-12-11 19:30:13 UTC (rev 165) @@ -0,0 +1,98 @@ +/* + * #%L + * EchoBase :: Tools + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ +package fr.ifremer.echobase.tools.loaders; + +import fr.ifremer.echobase.entities.data.Cell; +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; + +import java.util.Arrays; +import java.util.List; + +/** + * To test {@link CellLoader}. + * + * @author tchemit <chemit(a)codelutin.com> + * @since 0.2 + */ +(a)RunWith(Parameterized.class) +public class CellByDataBadProcessingLoaderTest extends AbstractLoaderTest<Cell, CellLoader> { + + /** Id of data processing to treat. */ + private String dataProcessingId; + + @Parameterized.Parameters + public static List<Object[]> data() { + return Arrays.asList(new Object[][]{ + {"EIT_0510_38"}, + {"EIT_83A_38"}, + {"EIT_83B_38"}, + {"EIT_84_38"}, + {"EIT_85_38"}, + {"EIT_86_38"}, + {"EIT_87_38"}, + {"EIT_89A_38"}, + {"EIT_89B_38"}, + {"EIT_90_38"}, + {"EIT_91_38"}, + {"EIT_92_38"}, + {"EIT_93_38"}, + {"EIT_94_38"}, + {"EIT_97_38"}, + }); + } + + static boolean cacheLoaded; + + @Override + public void setUp() throws Exception { + + loadDependencies = false; + + super.setUp(); + + if (!cacheLoaded) { + + // load caches once for all + + loader.loadAllProcessingIdsFromScenario(getDb()); + + cacheLoaded = true; + } + } + + public CellByDataBadProcessingLoaderTest(String dataProcessingId) { + super(CellLoader.class, 0, 0); + this.dataProcessingId = dataProcessingId; + } + + @Test + public void testDataProcessingBad() throws Exception { + boolean contains = loader.getAllProcessingIds().contains(dataProcessingId); + Assert.assertFalse(contains); + } + +} Property changes on: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellByDataBadProcessingLoaderTest.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Copied: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellByDataProcessingLoaderTest.java (from rev 163, trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellLoaderByDataProcessingLoaderTest.java) =================================================================== --- trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellByDataProcessingLoaderTest.java (rev 0) +++ trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellByDataProcessingLoaderTest.java 2011-12-11 19:30:13 UTC (rev 165) @@ -0,0 +1,236 @@ +/* + * #%L + * EchoBase :: Tools + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ +package fr.ifremer.echobase.tools.loaders; + +import com.google.common.base.Function; +import com.google.common.collect.Collections2; +import com.google.common.collect.Lists; +import com.healthmarketscience.jackcess.Database; +import fr.ifremer.echobase.EchoBaseIOUtil; +import fr.ifremer.echobase.entities.EchoBaseEntityEnum; +import fr.ifremer.echobase.entities.data.Cell; +import fr.ifremer.echobase.entities.data.DataProcessing; +import fr.ifremer.echobase.entities.meta.AssociationMeta; +import fr.ifremer.echobase.entities.meta.MetaFilenameAware; +import fr.ifremer.echobase.entities.meta.TableMeta; +import fr.ifremer.echobase.services.DbEditorService; +import fr.ifremer.echobase.tools.FileType; +import fr.ifremer.echobase.tools.TestHelper; +import org.apache.commons.collections.CollectionUtils; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.AfterClass; +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; +import org.nuiton.topia.TopiaException; + +import java.io.File; +import java.io.IOException; +import java.text.ParseException; +import java.util.Arrays; +import java.util.Collection; +import java.util.List; + +/** + * To test {@link CellLoader}. + * + * @author tchemit <chemit(a)codelutin.com> + * @since 0.2 + */ +(a)RunWith(Parameterized.class) +public class CellByDataProcessingLoaderTest extends AbstractLoaderTest<Cell, CellLoader> { + + /** Logger. */ + private static final Log log = + LogFactory.getLog(CellByDataProcessingLoaderTest.class); + + /** Id of data processing to treat. */ + private String dataProcessingId; + + @Parameterized.Parameters + public static List<Object[]> data() { + return Arrays.asList(new Object[][]{ + {"EIT_00_38", 0}, + {"EIT_01_38", 0}, + {"EIT_02_38", 0}, + {"EIT_03_38", 0}, + {"EIT_0310_38", 0}, + {"EIT_04_38", 0}, + {"EIT_05_38", 0}, +// {"EIT_0510_38", 0}, + {"EIT_06_38", 0}, + {"EIT_07_38", 0}, + {"EIT_08_38", 0}, + {"EIT_09_38", 0}, + {"EIT_10_38", 0}, +// {"EIT_83A_38", 0}, +// {"EIT_83B_38", 0}, +// {"EIT_84_38", 0}, +// {"EIT_85_38", 0}, +// {"EIT_86_38", 0}, +// {"EIT_87_38", 0}, +// {"EIT_89A_38", 0}, +// {"EIT_89B_38", 0}, +// {"EIT_90_38", 0}, +// {"EIT_91_38", 0}, +// {"EIT_92_38", 0}, +// {"EIT_93_38", 0}, +// {"EIT_94_38", 0}, +// {"EIT_97_38", 0}, + {"EIT_98b_38", 0}, + }); + } + + static boolean cacheLoaded; + + static List<File> directoriesToMerge = Lists.newLinkedList(); + + @Override + public void setUp() throws Exception { + + super.setUp(); + + if (!cacheLoaded) { + + // load caches once for all + + loader.loadCache(getDb()); + + cacheLoaded = true; + } + + Assert.assertTrue(loader.getAllProcessingIds().contains(dataProcessingId)); + } + + static CellLoader lastLoader; + + @Override + public void tearDown() throws Exception { + lastLoader = loader; + super.tearDown(); + + if (log.isInfoEnabled()) { + log.info("TimeLogs : " + DbEditorService.timeLog.getCallCount()); + } + } + + @AfterClass + public static void afterClass() throws Exception { + + // merge all results + if (CollectionUtils.isNotEmpty(directoriesToMerge)) { + + // let's merge all result + File dir = TestHelper.getTestBasedir(CellByDataProcessingLoaderTest.class); + + TableMeta cellMeta = lastLoader.getMeta(); + TableMeta dataMeta = lastLoader.getTable(EchoBaseEntityEnum.Data); + TableMeta dataProcessingMeta = lastLoader.getTable(EchoBaseEntityEnum.DataProcessing); + AssociationMeta dataProcessingToCellMetaAssociation = dataProcessingMeta.getAssociations(DataProcessing.PROPERTY_CELL); + AssociationMeta cellToCellMetaAssociation = cellMeta.getAssociations(Cell.PROPERTY_CHILDS); + AssociationMeta cellToDataMetaAssociation = cellMeta.getAssociations(Cell.PROPERTY_DATA); + + ToFileFromMeta toCellFile = new ToFileFromMeta(lastLoader, cellMeta); + ToFileFromMeta toDataFile = new ToFileFromMeta(lastLoader, dataMeta); + ToFileFromMeta toDataProcessingToCellFile = new ToFileFromMeta(lastLoader, dataProcessingToCellMetaAssociation); + ToFileFromMeta toCellToCellFile = new ToFileFromMeta(lastLoader, cellToCellMetaAssociation); + ToFileFromMeta toCellToDataFile = new ToFileFromMeta(lastLoader, cellToDataMetaAssociation); + + if (log.isInfoEnabled()) { + log.info("Will merge " + directoriesToMerge.size() + " data processing data into " + dir); + } + + mergeFile(dir, toCellFile); + mergeFile(dir, toDataFile); + mergeFile(dir, toDataProcessingToCellFile); + mergeFile(dir, toCellToCellFile); + mergeFile(dir, toCellToDataFile); + } + } + + protected static void mergeFile(File dir, ToFileFromMeta function) throws Exception { + + File mergedFile = function.apply(dir); + if (log.isInfoEnabled()) { + log.info("Will merge to " + mergedFile.getName()); + } + + Collection<File> files = Collections2.transform(directoriesToMerge, function); + + EchoBaseIOUtil.mergeFiles(mergedFile, files); + } + + public CellByDataProcessingLoaderTest(String dataProcessingId, int exceptedDataCount) { + super(CellLoader.class, 0, exceptedDataCount); + this.dataProcessingId = dataProcessingId; + } + + @Test + public void testGenerateEntitiesFromBaracouda() throws Exception { + generateBaracoudaEntities(); + } + + @Override + protected void launchEntitiesFromBaracouda(CellLoader loader, + Database db, + File exportdir) throws IOException, TopiaException, ParseException { + + if (log.isInfoEnabled()) { + log.info("Export to " + exportdir); + } + directoriesToMerge.add(exportdir); + loader.generateEntitiesFromBaracouda(db, exportdir, dataProcessingId, true); + } + + @Override + protected void importEntities(File inFile, + long nbExpected) throws IOException, TopiaException { + // do not use it + } + + private static class ToFileFromMeta implements Function<File, File> { + + private MetaFilenameAware meta; + private FileType fileType; + private CellLoader loader; + + public ToFileFromMeta(CellLoader loader, MetaFilenameAware meta) { + this.meta = meta; + this.loader = loader; + fileType = meta instanceof AssociationMeta ? FileType.ASSOCIATION : FileType.BARACOUDA; + } + + @Override + public File apply(File input) { + + try { + return loader.getTableFile(input, meta, fileType); + } catch (IOException e) { + throw new IllegalStateException("Could not obtain table file", e); + } + } + } +} Property changes on: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellByDataProcessingLoaderTest.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Deleted: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellLoaderByDataBadProcessingTest.java =================================================================== --- trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellLoaderByDataBadProcessingTest.java 2011-12-11 18:56:22 UTC (rev 164) +++ trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellLoaderByDataBadProcessingTest.java 2011-12-11 19:30:13 UTC (rev 165) @@ -1,98 +0,0 @@ -/* - * #%L - * EchoBase :: Tools - * - * $Id$ - * $HeadURL$ - * %% - * Copyright (C) 2011 Ifremer, Codelutin - * %% - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU Affero General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see <http://www.gnu.org/licenses/>. - * #L% - */ -package fr.ifremer.echobase.tools.loaders; - -import fr.ifremer.echobase.entities.data.Cell; -import org.junit.Assert; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.junit.runners.Parameterized; - -import java.util.Arrays; -import java.util.List; - -/** - * To test {@link CellLoader}. - * - * @author tchemit <chemit(a)codelutin.com> - * @since 0.2 - */ -(a)RunWith(Parameterized.class) -public class CellLoaderByDataBadProcessingTest extends AbstractLoaderTest<Cell, CellLoader> { - - /** Id of data processing to treat. */ - private String dataProcessingId; - - @Parameterized.Parameters - public static List<Object[]> data() { - return Arrays.asList(new Object[][]{ - {"EIT_0510_38"}, - {"EIT_83A_38"}, - {"EIT_83B_38"}, - {"EIT_84_38"}, - {"EIT_85_38"}, - {"EIT_86_38"}, - {"EIT_87_38"}, - {"EIT_89A_38"}, - {"EIT_89B_38"}, - {"EIT_90_38"}, - {"EIT_91_38"}, - {"EIT_92_38"}, - {"EIT_93_38"}, - {"EIT_94_38"}, - {"EIT_97_38"}, - }); - } - - static boolean cacheLoaded; - - @Override - public void setUp() throws Exception { - - loadDependencies = false; - - super.setUp(); - - if (!cacheLoaded) { - - // load caches once for all - - loader.loadAllProcessingIdsFromScenario(getDb()); - - cacheLoaded = true; - } - } - - public CellLoaderByDataBadProcessingTest(String dataProcessingId) { - super(CellLoader.class, 0, 0); - this.dataProcessingId = dataProcessingId; - } - - @Test - public void testDataProcessingBad() throws Exception { - boolean contains = loader.getAllProcessingIds().contains(dataProcessingId); - Assert.assertFalse(contains); - } - -} Deleted: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellLoaderByDataProcessingLoaderTest.java =================================================================== --- trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellLoaderByDataProcessingLoaderTest.java 2011-12-11 18:56:22 UTC (rev 164) +++ trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/CellLoaderByDataProcessingLoaderTest.java 2011-12-11 19:30:13 UTC (rev 165) @@ -1,236 +0,0 @@ -/* - * #%L - * EchoBase :: Tools - * - * $Id$ - * $HeadURL$ - * %% - * Copyright (C) 2011 Ifremer, Codelutin - * %% - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU Affero General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see <http://www.gnu.org/licenses/>. - * #L% - */ -package fr.ifremer.echobase.tools.loaders; - -import com.google.common.base.Function; -import com.google.common.collect.Collections2; -import com.google.common.collect.Lists; -import com.healthmarketscience.jackcess.Database; -import fr.ifremer.echobase.EchoBaseIOUtil; -import fr.ifremer.echobase.entities.EchoBaseEntityEnum; -import fr.ifremer.echobase.entities.data.Cell; -import fr.ifremer.echobase.entities.data.DataProcessing; -import fr.ifremer.echobase.entities.meta.AssociationMeta; -import fr.ifremer.echobase.entities.meta.MetaFilenameAware; -import fr.ifremer.echobase.entities.meta.TableMeta; -import fr.ifremer.echobase.services.DbEditorService; -import fr.ifremer.echobase.tools.FileType; -import fr.ifremer.echobase.tools.TestHelper; -import org.apache.commons.collections.CollectionUtils; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.junit.AfterClass; -import org.junit.Assert; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.junit.runners.Parameterized; -import org.nuiton.topia.TopiaException; - -import java.io.File; -import java.io.IOException; -import java.text.ParseException; -import java.util.Arrays; -import java.util.Collection; -import java.util.List; - -/** - * To test {@link CellLoader}. - * - * @author tchemit <chemit(a)codelutin.com> - * @since 0.2 - */ -(a)RunWith(Parameterized.class) -public class CellLoaderByDataProcessingLoaderTest extends AbstractLoaderTest<Cell, CellLoader> { - - /** Logger. */ - private static final Log log = - LogFactory.getLog(CellLoaderByDataProcessingLoaderTest.class); - - /** Id of data processing to treat. */ - private String dataProcessingId; - - @Parameterized.Parameters - public static List<Object[]> data() { - return Arrays.asList(new Object[][]{ - {"EIT_00_38", 0}, - {"EIT_01_38", 0}, - {"EIT_02_38", 0}, - {"EIT_03_38", 0}, - {"EIT_0310_38", 0}, - {"EIT_04_38", 0}, - {"EIT_05_38", 0}, -// {"EIT_0510_38", 0}, - {"EIT_06_38", 0}, - {"EIT_07_38", 0}, - {"EIT_08_38", 0}, - {"EIT_09_38", 0}, - {"EIT_10_38", 0}, -// {"EIT_83A_38", 0}, -// {"EIT_83B_38", 0}, -// {"EIT_84_38", 0}, -// {"EIT_85_38", 0}, -// {"EIT_86_38", 0}, -// {"EIT_87_38", 0}, -// {"EIT_89A_38", 0}, -// {"EIT_89B_38", 0}, -// {"EIT_90_38", 0}, -// {"EIT_91_38", 0}, -// {"EIT_92_38", 0}, -// {"EIT_93_38", 0}, -// {"EIT_94_38", 0}, -// {"EIT_97_38", 0}, - {"EIT_98b_38", 0}, - }); - } - - static boolean cacheLoaded; - - static List<File> directoriesToMerge = Lists.newLinkedList(); - - @Override - public void setUp() throws Exception { - - super.setUp(); - - if (!cacheLoaded) { - - // load caches once for all - - loader.loadCache(getDb()); - - cacheLoaded = true; - } - - Assert.assertTrue(loader.getAllProcessingIds().contains(dataProcessingId)); - } - - static CellLoader lastLoader; - - @Override - public void tearDown() throws Exception { - lastLoader = loader; - super.tearDown(); - - if (log.isInfoEnabled()) { - log.info("TimeLogs : " + DbEditorService.timeLog.getCallCount()); - } - } - - @AfterClass - public static void afterClass() throws Exception { - - // merge all results - if (CollectionUtils.isNotEmpty(directoriesToMerge)) { - - // let's merge all result - File dir = TestHelper.getTestBasedir(CellLoaderByDataProcessingLoaderTest.class); - - TableMeta cellMeta = lastLoader.getMeta(); - TableMeta dataMeta = lastLoader.getTable(EchoBaseEntityEnum.Data); - TableMeta dataProcessingMeta = lastLoader.getTable(EchoBaseEntityEnum.DataProcessing); - AssociationMeta dataProcessingToCellMetaAssociation = dataProcessingMeta.getAssociations(DataProcessing.PROPERTY_CELL); - AssociationMeta cellToCellMetaAssociation = cellMeta.getAssociations(Cell.PROPERTY_CHILDS); - AssociationMeta cellToDataMetaAssociation = cellMeta.getAssociations(Cell.PROPERTY_DATA); - - ToFileFromMeta toCellFile = new ToFileFromMeta(lastLoader, cellMeta); - ToFileFromMeta toDataFile = new ToFileFromMeta(lastLoader, dataMeta); - ToFileFromMeta toDataProcessingToCellFile = new ToFileFromMeta(lastLoader, dataProcessingToCellMetaAssociation); - ToFileFromMeta toCellToCellFile = new ToFileFromMeta(lastLoader, cellToCellMetaAssociation); - ToFileFromMeta toCellToDataFile = new ToFileFromMeta(lastLoader, cellToDataMetaAssociation); - - if (log.isInfoEnabled()) { - log.info("Will merge " + directoriesToMerge.size() + " data processing data into " + dir); - } - - mergeFile(dir, toCellFile); - mergeFile(dir, toDataFile); - mergeFile(dir, toDataProcessingToCellFile); - mergeFile(dir, toCellToCellFile); - mergeFile(dir, toCellToDataFile); - } - } - - protected static void mergeFile(File dir, ToFileFromMeta function) throws Exception { - - File mergedFile = function.apply(dir); - if (log.isInfoEnabled()) { - log.info("Will merge to " + mergedFile.getName()); - } - - Collection<File> files = Collections2.transform(directoriesToMerge, function); - - EchoBaseIOUtil.mergeFiles(mergedFile, files); - } - - public CellLoaderByDataProcessingLoaderTest(String dataProcessingId, int exceptedDataCount) { - super(CellLoader.class, 0, exceptedDataCount); - this.dataProcessingId = dataProcessingId; - } - - @Test - public void testGenerateEntitiesFromBaracouda() throws Exception { - generateBaracoudaEntities(); - } - - @Override - protected void launchEntitiesFromBaracouda(CellLoader loader, - Database db, - File exportdir) throws IOException, TopiaException, ParseException { - - if (log.isInfoEnabled()) { - log.info("Export to " + exportdir); - } - directoriesToMerge.add(exportdir); - loader.generateEntitiesFromBaracouda(db, exportdir, dataProcessingId, true); - } - - @Override - protected void importEntities(File inFile, - long nbExpected) throws IOException, TopiaException { - // do not use it - } - - private static class ToFileFromMeta implements Function<File, File> { - - private MetaFilenameAware meta; - private FileType fileType; - private CellLoader loader; - - public ToFileFromMeta(CellLoader loader, MetaFilenameAware meta) { - this.meta = meta; - this.loader = loader; - fileType = meta instanceof AssociationMeta ? FileType.ASSOCIATION : FileType.BARACOUDA; - } - - @Override - public File apply(File input) { - - try { - return loader.getTableFile(input, meta, fileType); - } catch (IOException e) { - throw new IllegalStateException("Could not obtain table file", e); - } - } - } -}

1 0