Skip to content

Commit

Permalink
hdfs bundle for 23.4 (#350)
Browse files Browse the repository at this point in the history
# Description

Bundle files for the 23.4 release of the hdfs operator.

Tests when deployed from bundle:on 4.11:
```
--- PASS: kuttl (5784.18s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/cluster-operation_hadoop-latest-3.3.4-stackable23.4.0_zookeeper-latest-3.8.0-stackable23.4.0 (284.03s)
        --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (169.42s)
        --- PASS: kuttl/harness/logging_hadoop-3.3.4-stackable23.4.0_zookeeper-latest-3.8.0-stackable23.4.0 (294.27s)
        --- PASS: kuttl/harness/logging_hadoop-3.3.3-stackable23.4.0_zookeeper-latest-3.8.0-stackable23.4.0 (303.57s)
        --- PASS: kuttl/harness/logging_hadoop-3.2.2-stackable23.4.0_zookeeper-latest-3.8.0-stackable23.4.0 (254.94s)
        --- PASS: kuttl/harness/orphaned-resources_hadoop-latest-3.3.4-stackable23.4.0_zookeeper-latest-3.8.0-stackable23.4.0 (155.49s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-external-unstable (203.30s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-cluster-internal (223.24s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-cluster-internal (175.53s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (155.69s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (175.98s)
        --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-external-unstable (169.14s)
        --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-cluster-internal (200.51s)
        --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (207.47s)
        --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (197.76s)
        --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-external-unstable (161.26s)
        --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-cluster-internal (181.51s)
        --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (165.62s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (160.48s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (207.90s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (219.98s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-external-unstable (155.46s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-external-unstable (156.55s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (174.96s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-cluster-internal (149.32s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (210.23s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-cluster-internal (185.76s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (244.39s)
        --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-external-unstable (230.97s)
```




Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>
  • Loading branch information
adwk67 and razvan committed May 4, 2023
1 parent f8c4b04 commit d40b632
Show file tree
Hide file tree
Showing 8 changed files with 4,740 additions and 1 deletion.
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,7 @@ crate-hashes.json
result
image.tar

tilt_options.json
tilt_options.json

**/bundle/
**/bundle.Dockerfile
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,18 @@ All notable changes to this project will be documented in this file.

## [Unreleased]

### Added

- Generate OLM bundle for Release 23.4.0 ([#350]).

### Changed

- Operator-rs: `0.40.2` -> `0.41.0` ([#349]).
- Use 0.0.0-dev product images for testing ([#351])
- Use testing-tools 0.2.0 ([#351])

[#349]: https://github.com/stackabletech/hdfs-operator/pull/349
[#350]: https://github.com/stackabletech/hdfs-operator/pull/350
[#351]: https://github.com/stackabletech/hdfs-operator/pull/351

## [23.4.0] - 2023-04-17
Expand Down
177 changes: 177 additions & 0 deletions deploy/olm/23.4.0/manifests/configmap.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
---
apiVersion: v1
data:
properties.yaml: |
version: 0.1.0
spec:
units:
- unit: &unitUri
name: "uri"
regex: "^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\\?([^#]*))?(#(.*))?"
examples:
- "ldap://[2001:db8::7]/c=GB?objectClass?one"
comment: "Specified in https://tools.ietf.org/html/rfc3986#appendix-B"
properties:
- property:
propertyNames:
- name: "dfs.namenode.name.dir"
kind:
type: "file"
file: "hdfs-site.xml"
datatype:
type: "string"
unit: *unitUri
roles:
- name: "namenode"
required: false
- name: "datanode"
required: false
asOfVersion: "0.0.0"
description: "Determines where on the local filesystem the DFS name node should store the name table(fsimage). If this is a comma-delimited list of directories then the name table is replicated in all of the directories, for redundancy."
- property:
propertyNames:
- name: "dfs.datanode.data.dir"
kind:
type: "file"
file: "hdfs-site.xml"
datatype:
type: "string"
unit: *unitUri
roles:
- name: "namenode"
required: false
- name: "datanode"
required: false
asOfVersion: "0.0.0"
description: "Determines where on the local filesystem an DFS data node should store its blocks. If this is a comma-delimited list of directories, then data will be stored in all named directories, typically on different devices. Directories that do not exist are ignored."
- property:
propertyNames:
- name: "dfs.replication"
kind:
type: "file"
file: "hdfs-site.xml"
datatype:
type: "integer"
min: "1"
max: "512"
defaultValues:
- fromVersion: "0.0.0"
value: "3"
roles:
- name: "namenode"
required: true
- name: "datanode"
required: true
asOfVersion: "0.0.0"
description: "Default block replication. The actual number of replications can be specified when the file is created. The default is used if replication is not specified in create time."
- property:
propertyNames:
- name: "fs.defaultFS"
kind:
type: "file"
file: "core-site.xml"
datatype:
type: "string"
unit: *unitUri
roles:
- name: "namenode"
required: false
- name: "datanode"
required: false
asOfVersion: "0.0.0"
description: "The name of the default file system. A URI whose scheme and authority determine the FileSystem implementation. The uri's scheme determines the config property (fs.SCHEME.impl) naming the FileSystem implementation class. The uri's authority is used to determine the host, port, etc. for a filesystem."
- property:
propertyNames:
- name: "dfs.namenode.http-address"
kind:
type: "file"
file: "hdfs-site.xml"
datatype:
type: "string"
roles:
- name: "namenode"
required: false
asOfVersion: "0.0.0"
description: "The address and the base port where the dfs namenode web ui will listen on."
- property:
propertyNames:
- name: "dfs.datanode.ipc.address"
kind:
type: "file"
file: "hdfs-site.xml"
datatype:
type: "string"
roles:
- name: "datanode"
required: false
asOfVersion: "0.0.0"
description: "The datanode ipc server address and port."
- property:
propertyNames:
- name: "dfs.datanode.address"
kind:
type: "file"
file: "hdfs-site.xml"
datatype:
type: "string"
roles:
- name: "datanode"
required: false
asOfVersion: "0.0.0"
description: "The datanode server address and port for data transfer."
- property:
propertyNames:
- name: "dfs.datanode.http.address"
kind:
type: "file"
file: "hdfs-site.xml"
datatype:
type: "string"
roles:
- name: "datanode"
required: false
asOfVersion: "0.0.0"
description: "The datanode http server address and port."
- property:
propertyNames:
- name: "dfs.journalnode.http-address"
kind:
type: "file"
file: "hdfs-site.xml"
datatype:
type: "string"
roles:
- name: "journalnode"
required: false
asOfVersion: "0.0.0"
description: "The address and port the JournalNode HTTP server listens on. If the port is 0 then the server will start on a free port."
- property:
propertyNames:
- name: "dfs.journalnode.https-address"
kind:
type: "file"
file: "hdfs-site.xml"
datatype:
type: "string"
roles:
- name: "journalnode"
required: false
asOfVersion: "0.0.0"
description: "The address and port the JournalNode HTTPS server listens on. If the port is 0 then the server will start on a free port."
kind: ConfigMap
metadata:
name: hdfs-operator-configmap
labels:
app.kubernetes.io/name: hdfs-operator
app.kubernetes.io/instance: hdfs-operator
app.kubernetes.io/version: "23.4.0"
Loading

0 comments on commit d40b632

Please sign in to comment.