Harvester / Simple URL #4034

fxprunayre · 2019-09-17T07:37:39Z

A simple harvester which takes a URL expecting for now a JSON document
and loop over document identified by a JSONPointer and applying an XSL
to convert to ISO format.

This should allow GeoNetwork to harvest some of the opendata portal
providing all various search API providing JSON response usually.

A simple harvester which takes a URL expecting for now a JSON document and loop over document identified by a JSONPointer and applying an XSL to convert to ISO format. This should allow GeoNetwork to harvest some of the opendata portal providing all various search API providing JSON response usually.

fxprunayre · 2019-12-16T10:11:00Z

Does not look to have much interest on this. Closing.

archaeogeek · 2021-01-11T14:12:21Z

@fxprunayre Testing this with 3.10.x but I think I'm getting a build problem relating to:

Error: 1,683 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project harvesters: Compilation failure: Compilation failure: 
Error: 1,683 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[46,58] wrong number of type arguments; required 2
Error: 8,221 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[46,58] wrong number of type arguments; required 2
Error: 8,221 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlParams.java:[32,8] org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlParams is not abstract and does not override abstract method getIcon() in org.fao.geonet.kernel.harvest.harvester.AbstractParams
Error: 8,222 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[54,38] cannot find symbol
  symbol:   variable dataMan
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,222 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[55,9] cannot find symbol
  symbol:   variable super
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,222 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[60,38] cannot find symbol
  symbol:   variable dataMan
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,223 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[61,9] cannot find symbol
  symbol:   variable super
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,223 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[66,72] cannot find symbol
  symbol:   method getType()
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,223 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[66,21] cannot find symbol
  symbol:   variable harvesterSettingsManager
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,224 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[68,9] cannot find symbol
  symbol:   method storeNode(org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlParams,java.lang.String)
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,224 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[73,28] cannot find symbol
  symbol:   variable context
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,225 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[73,18] non-static method copyLogo(jeeves.server.context.ServiceContext,java.lang.String,java.lang.String) cannot be referenced from a static context
Error: 8,225 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[80,9] cannot find symbol
  symbol:   variable super
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,230 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[85,9] cannot find symbol
  symbol:   variable harvesterSettingsManager
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,230 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[87,9] cannot find symbol
  symbol:   method storeNode(org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlParams,java.lang.String)
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,231 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[92,28] cannot find symbol
  symbol:   variable context
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,231 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[92,18] non-static method copyLogo(jeeves.server.context.ServiceContext,java.lang.String,java.lang.String) cannot be referenced from a static context
Error: 8,231 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[96,9] cannot find symbol
  symbol:   variable super
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,232 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[111,9] cannot find symbol
  symbol:   variable harvesterSettingsManager
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,232 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[112,9] cannot find symbol
  symbol:   variable harvesterSettingsManager
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,233 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[113,9] cannot find symbol
  symbol:   variable harvesterSettingsManager
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,233 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[114,9] cannot find symbol
  symbol:   variable harvesterSettingsManager
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,233 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[115,9] cannot find symbol
  symbol:   variable harvesterSettingsManager
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,234 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[116,9] cannot find symbol
  symbol:   variable harvesterSettingsManager
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,234 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[117,9] cannot find symbol
  symbol:   variable harvesterSettingsManager
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,234 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[118,9] cannot find symbol
  symbol:   variable harvesterSettingsManager
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,235 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[122,37] cannot find symbol
  symbol:   variable cancelMonitor
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,235 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[122,57] cannot find symbol
  symbol:   variable context
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,235 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/SimpleUrlHarvester.java:[123,9] cannot find symbol
  symbol:   variable result
  location: class org.fao.geonet.kernel.harvest.harvester.simpleUrl.SimpleUrlHarvester
Error: 8,236 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/Aligner.java:[241,9] method addPrivileges in class org.fao.geonet.kernel.harvest.BaseAligner<P> cannot be applied to given types;
  required: java.lang.String,java.lang.Iterable<org.fao.geonet.kernel.harvest.harvester.Privileges>,org.fao.geonet.kernel.harvest.harvester.GroupMapper,jeeves.server.context.ServiceContext
  found: java.lang.String,java.lang.Iterable<org.fao.geonet.kernel.harvest.harvester.Privileges>,org.fao.geonet.kernel.harvest.harvester.GroupMapper,org.fao.geonet.kernel.DataManager,jeeves.server.context.ServiceContext
  reason: actual and formal argument lists differ in length
Error: 8,246 [ERROR] /home/runner/work/custom-geonetwork/custom-geonetwork/harvesters/src/main/java/org/fao/geonet/kernel/harvest/harvester/simpleUrl/Aligner.java:[276,9] method addPrivileges in class org.fao.geonet.kernel.harvest.BaseAligner<P> cannot be applied to given types;
  required: java.lang.String,java.lang.Iterable<org.fao.geonet.kernel.harvest.harvester.Privileges>,org.fao.geonet.kernel.harvest.harvester.GroupMapper,jeeves.server.context.ServiceContext
  found: java.lang.String,java.lang.Iterable<org.fao.geonet.kernel.harvest.harvester.Privileges>,org.fao.geonet.kernel.harvest.harvester.GroupMapper,org.fao.geonet.kernel.DataManager,jeeves.server.context.ServiceContext
  reason: actual and formal argument lists differ in length

I cherry-picked the PR and merged it into a clean 3.10.x, then committed it to my repository, then the Linux GitHub CI workflow reported this error.

Is there more that I need to do to get this to work?

Thanks

Jo

fxprunayre · 2021-04-13T10:25:47Z

No resource here to progress on this. Closing for now.

fxprunayre added this to the 3.8.1 milestone Sep 17, 2019

Harvester / Simple URL / Paging and basic opendatasoft support.

ed05387

fxprunayre modified the milestones: 3.8.1, 3.8.2 Sep 18, 2019

fxprunayre mentioned this pull request Oct 20, 2019

geonetwork - csw harvester does not provide any UI to filter incoming metadata georchestra/georchestra#2792

Closed

fxprunayre modified the milestones: 3.8.2, 3.8.3 Oct 29, 2019

fxprunayre closed this Dec 16, 2019

fxprunayre mentioned this pull request Mar 24, 2020

Harvest ESRI Open Data via DCAT #4541

Open

archaeogeek reopened this Jan 11, 2021

fxprunayre mentioned this pull request Feb 15, 2021

ogcapi-Records harvester #5426

Closed

fxprunayre closed this Apr 13, 2021

fgravin mentioned this pull request Sep 6, 2021

Json harvester #5942

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Harvester / Simple URL #4034

Harvester / Simple URL #4034

fxprunayre commented Sep 17, 2019

fxprunayre commented Dec 16, 2019

archaeogeek commented Jan 11, 2021

fxprunayre commented Apr 13, 2021

Harvester / Simple URL #4034

Harvester / Simple URL #4034

Conversation

fxprunayre commented Sep 17, 2019

fxprunayre commented Dec 16, 2019

archaeogeek commented Jan 11, 2021

fxprunayre commented Apr 13, 2021