Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fixes #160] Bug: Error in tasks.py when initiating GeoNode container #161

Merged
merged 6 commits into from
Mar 8, 2024
+29 −40
Merged

[Fixes #160] Bug: Error in tasks.py when initiating GeoNode container #161

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
43a3286
[Fixes #160] Bug: Error in tasks.py when initiating GeoNode container
mwallschlaeger Mar 7, 2024
2b17632
[Fixes #160] Bug: Error in tasks.py when initiating GeoNode container
mwallschlaeger Mar 7, 2024
d63438c
[Fixes #160] Bug: Error in tasks.py when initiating GeoNode container
Mar 7, 2024
c8a546b
Merge branch 'main' into issue_#160_Bug_Error_in_tasks.py_when_initia…
mwallschlaeger Mar 7, 2024
f0214b3
[Fixes #160] Bug: Error in tasks.py when initiating GeoNode container
mwallschlaeger Mar 7, 2024
6987c72
Merge branch 'main' into issue_#160_Bug_Error_in_tasks.py_when_initia…
mwallschlaeger Mar 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions charts/geonode/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,8 @@ Helm Chart for Geonode. Supported versions: Geonode: 4.1.3, Geoserver: 2.23.0, p
| geoserver.resources.limits.memory | string | `"4Gi"` | limits memory as in resource.limits.memory (https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
| geoserver.resources.requests.cpu | int | `1` | requested cpu as in resource.requests.cpu (https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
| geoserver.resources.requests.memory | string | `"1Gi"` | requested memory as in resource.requests.memory (https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
| geoserver.secret.admin_password | string | `"geoserver"` | geoserver admin password |
| geoserver.secret.admin_factory_password | string | `"geoserver"` | geoserver factory/previous password. This password is used to change the current password for the geoserver admin_user |
| geoserver.secret.admin_password | string | `"geoserver"` | geoserver admin password, respect admin_factory_password, if you gonna change the password from not factory password |
| geoserver.secret.admin_username | string | `"admin"` | geoserver admin username |
| geoserver.secret.existingSecretName | string | `""` | name of an existing Secret to use. Set, if you want to separately maintain the Secret. |
| geoserver_data.container_name | string | `"geoserver-data-dir"` | |
Expand Down Expand Up @@ -217,4 +218,4 @@ Helm Chart for Geonode. Supported versions: Geonode: 4.1.3, Geoserver: 2.23.0, p
| rabbitmq.requests.memory | string | `"1Gi"` | requested memory as in resource.requests.memory (https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |

----------------------------------------------
Autogenerated from chart metadata using [helm-docs v1.11.0](https://github.com/norwoodj/helm-docs/releases/v1.11.0)
Autogenerated from chart metadata using [helm-docs v1.13.1](https://github.com/norwoodj/helm-docs/releases/v1.13.1)
1 change: 1 addition & 0 deletions charts/geonode/templates/geonode/geonode-deploy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ spec:
annotations:
checksum/geonode-env: {{ include (print $.Template.BasePath "/geonode/geonode-env.yaml") . | sha256sum }}
checksum/geonode-secret: {{ include (print $.Template.BasePath "/geonode/geonode-secret.yaml") . | sha256sum }}
checksum/geonode-tasks-py: {{ include (print $.Template.BasePath "/geonode/geonode-tasks-py-conf.yaml") . | sha256sum }}
checksum/geonode-local-settings: {{ include (print $.Template.BasePath "/geonode/geonode-local-settings.yaml") . | sha256sum }}
checksum/geonode-settings-additions: {{ include (print $.Template.BasePath "/geonode/geonode-settings-additions.yaml") . | sha256sum }}
checksum/geonode-uwsig: {{ include (print $.Template.BasePath "/geonode/geonode-uwsgi-ini-conf.yaml") . | sha256sum }}
Expand Down
49 changes: 14 additions & 35 deletions charts/geonode/templates/geonode/geonode-tasks-py-conf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,8 @@ data:

logger = logging.getLogger(__name__)

GEOSERVER_STORE_NAME = "geogeonode"
GEOSERVER_STORE_NAME = "geogeonode"
GEOSERVER_WORKSPACE_NAME = "geonode"
GEOSERVER_USERNAME = "{{ .Values.geoserver.admin_username }}"

@task
def waitfordbs(ctx):
Expand Down Expand Up @@ -366,7 +365,6 @@ data:
@task
def geoserverfixture(ctx):
print("********************geoserver fixture********************************")
_geoserver_info_provision(f"{os.environ['GEOSERVER_LOCATION']}rest/")
_set_geoserver_database_store()

@task
Expand Down Expand Up @@ -451,9 +449,9 @@ data:
settings = os.getenv('DJANGO_SETTINGS_MODULE', 'geonode.settings')
return settings


def _gs_service_availability(url):
import requests

try:
r = requests.request("get", url, verify=False)
r.raise_for_status() # Raises a HTTPError if the status is 4xx, 5xxx
Expand All @@ -468,26 +466,6 @@ data:
return True


def _geoserver_info_provision(url):
from django.conf import settings
from geoserver.catalog import Catalog

print("Setting GeoServer Admin Password...")
cat = Catalog(url, username=settings.OGC_SERVER_DEFAULT_USER, password=settings.OGC_SERVER_DEFAULT_PASSWORD)
headers = {"Content-type": "application/xml", "Accept": "application/xml"}
data = f"""<?xml version="1.0" encoding="UTF-8"?>
<userPassword>
<newPassword>{(os.getenv('GEOSERVER_ADMIN_PASSWORD', 'geoserver'))}</newPassword>
</userPassword>"""

response = cat.http_request(f"{cat.service_url}/security/self/password", method="PUT", data=data, headers=headers)
print(f"Response Code: {response.status_code}")
if response.status_code == 200:
print("GeoServer admin password updated SUCCESSFULLY!")
else:
logger.warning(f"WARNING: GeoServer admin password *NOT* updated: code [{response.status_code}]")


def _prepare_oauth_fixture():
upurl = urlparse(os.environ["SITEURL"])
default_fixture = [
Expand All @@ -512,6 +490,7 @@ data:
with open("/tmp/default_oauth_apps_docker.json", "w") as fixturefile:
json.dump(default_fixture, fixturefile)


def _prepare_site_fixture():
upurl = urlparse(os.environ["SITEURL"])
default_fixture = [
Expand All @@ -524,6 +503,7 @@ data:
with open("/tmp/default_site.json", "w") as fixturefile:
json.dump(default_fixture, fixturefile)


def _prepare_monitoring_fixture():
pub_ip = os.getenv('GEONODE_LB_HOST_IP')
print(f"Public Hostname or IP is {pub_ip}")
Expand Down Expand Up @@ -627,35 +607,34 @@ data:
with open("/tmp/django_admin_docker.json", "w") as fixturefile:
json.dump(default_fixture, fixturefile)


##############################################################
# added by mwall for https://github.com/zalf-rdm/geonode-k8s #
##############################################################

def _set_geoserver_database_store():
''' checks if a db store is already created in geoserver, if not create one '''

print("Check if geoserver store is already set up ...")
geoserver_base_url = os.getenv('GEOSERVER_LOCATION')
geoserver_password = os.getenv('GEOSERVER_ADMIN_PASSWORD')
url = '{}rest/workspaces/{}/datastores/'.format(geoserver_base_url,GEOSERVER_WORKSPACE_NAME)
url = '{}rest/workspaces/{}/datastores/'.format(os.getenv('GEOSERVER_LOCATION'), GEOSERVER_WORKSPACE_NAME)
headers = {'Content-Type': 'text/xml'}
auth = (GEOSERVER_USERNAME, geoserver_password)
auth = (os.environ.get("GEOSERVER_ADMIN_USER"), os.getenv('GEOSERVER_ADMIN_PASSWORD'))

try:
r = requests.get(url, headers=headers, auth=auth)
data = r.json()
for datastore in data["dataStores"]["dataStore"]:
if datastore["name"] == GEOSERVER_STORE_NAME:
print("Found datastore \"{}\" on geoserver, no need to create another ... ".format(GEOSERVER_STORE_NAME))
print("Found datastore \"{}\" on geoserver, skipping creation ... ".format(GEOSERVER_STORE_NAME))
return
except:
print("could not get datastore information from geoserver, trying to create ...")
print("ERROR: Could not get datastore information from geoserver, trying to create ...")

print("setup new datastore ...")
print("create new datastore \"{}\" on geoserver ...".format(GEOSERVER_STORE_NAME))
db_user = database = os.getenv('GEONODE_GEODATABASE')
db_password = os.getenv('GEONODE_GEODATABASE_PASSWORD')
db_host = os.getenv('DATABASE_HOST')
db_port = os.getenv('DATABASE_PORT')
url = '{}rest/workspaces/{}/datastores'.format(geoserver_base_url,GEOSERVER_WORKSPACE_NAME)
url = '{}rest/workspaces/{}/datastores'.format(os.getenv('GEOSERVER_LOCATION'), GEOSERVER_WORKSPACE_NAME)
data = """
<dataStore>
<name>{}</name>
Expand All @@ -671,7 +650,7 @@ data:
""".format(GEOSERVER_STORE_NAME, db_host, db_port, database, db_user, db_password)
try:
r = requests.post(url, headers=headers, auth=auth,data=data)
print("created ...")
print("Datastore created ...")
except:
print("something went wrong while performing datastore creating, skipping ...")
print("ERROR: Something went wrong while creating datastore ...")
return
1 change: 1 addition & 0 deletions charts/geonode/templates/geoserver/geoserver-env.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,4 @@ data:
DATABASE_PORT: "{{ include "database_port" . }}"
GEONODE_GEODATABASE: {{ .Values.postgres.geodata_databasename_and_username | quote }}
GEONODE_GEODATABASE_SCHEMA: {{ .Values.postgres.schema | quote }}
FORCE_REINIT: 'true'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if you really want to reconfigure GeoServer always .. TBH I do not know how geonode-k8s handles that flag on the other containers. It is used in the entrypoint.sh of geonode/celery containers at least.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ridoo for geoserver I don't think its a big deal. atm it only updates the password of the geoserver admin and updates the timestamp of the .lock file. And for celery its kind of the same, updating fixtures, updating the timestamp of the .lock file and updating the geonode admin password if it has changed.

In my opinion, except the .lock update everything is required at each pod start anyway, or at least dosn't destroys anything. Building a switch arround this variable would bring another layer of complexity into this chart. What i could image is to add an values.yaml parameter which is enabled by default but can be disabled like:

geonode:
  force_reinit: true

geoserver:
  force_reinit: true

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Completely fine with me 👍

2 changes: 1 addition & 1 deletion charts/geonode/templates/geoserver/geoserver-secret.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ data:
# geoserver admin credentials
GEOSERVER_ADMIN_USER: {{ .Values.geoserver.secret.admin_username | b64enc }}
GEOSERVER_ADMIN_PASSWORD: {{ .Values.geoserver.secret.admin_password | b64enc }}

GEOSERVER_FACTORY_PASSWORD: {{ .Values.geoserver.secret.admin_factory_password | b64enc }}
# geonode oauth2
OAUTH2_API_KEY: ""
OAUTH2_CLIENT_ID: {{ .Values.geonode.secret.oauth2.clientId | b64enc}}
Expand Down
6 changes: 4 additions & 2 deletions charts/geonode/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,10 +337,12 @@ geoserver:
secret:
# -- name of an existing Secret to use. Set, if you want to separately maintain the Secret.
existingSecretName: ""
# -- geoserver factory/previous password. This password is used to change the current password for the geoserver admin_user
admin_factory_password: geoserver
# -- geoserver admin username
admin_username: admin
# -- geoserver admin password
admin_password: "geoserver"
# -- geoserver admin password, respect admin_factory_password, if you gonna change the password from not factory password
admin_password: geoserver

resources:
requests:
Expand Down
Loading