Scale 5.0.4.3にS3 Objectノードをインストールする
Objectノードを構成する
NFSノードがインストールできたので、Objectノードをインストールする。
ノード構成は以下のとおり。
NSD Servers:
snode-0 192.168.1.91
snode-1 192.168.1.92
CES Nodes:
nfsgroup1 192.168.1.98 (CES IP) <- NFS access用
pnode-1 192.168.1.96
pnode-2 192.168.1.97
s3group1 192.168.1.108 (CES IP) <- S3 access用
pnode-s1 192.168.1.106
pnode-s2 192.168.1.107
見ての通り、CES Nodesは2つのGroupに分けて、それぞれをNFS、S3アクセス用に設定する。すべてのCES NodesにはNFSとOBJの両方のProtocol Serviceがインストールされる。CES Nodeに対して選択的にProtocolをインストールすることはできない。
pnode-s1とpnode-s2を新たに用意してClusterに追加する。
[root@snode-0 gpfsfs01]# mmaddnode -N pnode-s1,pnode-s2
2020年 4月 1日 水曜日 13:41:48 JST: mmaddnode: Processing node pnode-s1
2020年 4月 1日 水曜日 13:41:51 JST: mmaddnode: Processing node pnode-s2
mmaddnode: Command successfully completed
mmaddnode: Warning: Not all nodes have proper GPFS license designations.
Use the mmchlicense command to designate licenses as needed.
mmaddnode: Propagating the cluster configuration data to all
affected nodes. This is an asynchronous process.
[root@snode-0 gpfsfs01]#
ライセンスが設定されていないと注意を受けた。
[root@snode-0 gpfsfs01]# mmlsnode
===============================================================================| Warning: |
| This cluster contains nodes that do not have a proper GPFS license |
| designation. This violates the terms of the GPFS licensing agreement. |
| Use the mmchlicense command and assign the appropriate GPFS licenses |
| to each of the nodes in the cluster. For more information about GPFS |
| license designation, see the Concepts, Planning, and Installation Guide. |
===============================================================================
GPFS nodeset Node list
------------- -------------------------------------------------------
gpfscluster1 snode-1 snode-0 pnode-1 pnode-2 pnode-s1 pnode-s2
CES NodeはServer Licenseが必要なので、それを設定する。
[root@snode-0 gpfsfs01]# mmchlicense server --accept -N pnode-s1,pnode-s2
The following nodes will be designated as possessing server licenses:
pnode-s1
pnode-s2
mmchlicense: Command successfully completed
mmchlicense: Propagating the cluster configuration data to all
affected nodes. This is an asynchronous process.
とりあえず確認。
[root@snode-0 gpfsfs01]# mmlsnode
GPFS nodeset Node list
------------- -------------------------------------------------------
gpfscluster1 snode-1 snode-0 pnode-1 pnode-2 pnode-s1 pnode-s2
[root@snode-0 gpfsfs01]#
[root@snode-0 gpfsfs01]# mmlscluster
GPFS cluster information
========================
GPFS cluster name: gpfscluster1.snode-0
GPFS cluster id: 17390685069738965895
GPFS UID domain: gpfscluster1.snode-0
Remote shell command: /usr/bin/ssh
Remote file copy command: /usr/bin/scp
Repository type: CCR
Node Daemon node name IP address Admin node name Designation
---------------------------------------------------------------------
1 snode-0 192.168.1.91 snode-0 quorum-manager
2 snode-1 192.168.1.92 snode-1 quorum-manager
3 pnode-1 192.168.1.96 pnode-1
4 pnode-2 192.168.1.97 pnode-2
5 pnode-s1 192.168.1.106 pnode-s1
6 pnode-s2 192.168.1.107 pnode-s2
[root@snode-0 gpfsfs01]#
cesSharedRootを確認。これが意味を持つのだろうか。後から要確認。
[root@snode-0 gpfsfs01]# mmlsconfig
Configuration data for cluster gpfscluster1.snode-0:
----------------------------------------------------
clusterName gpfscluster1.snode-0
clusterId 17390685069738965895
autoload no
dmapiFileHandleSize 32
minReleaseLevel 5.0.4.0
ccrEnabled yes
cipherList AUTHONLY
cesSharedRoot /gpfs/gpfsfs01/nfs
adminMode central
File systems in cluster gpfscluster1.snode-0:
---------------------------------------------
/dev/gpfsfs01
現時点でのCES Nodeを確認。
[root@snode-0 /]# mmces node list
Node Number Node Name Node Groups Node Flags
------------- ----------- ------------- ------------
3 pnode-1 nfsgroup1 none
4 pnode-2 nfsgroup1 none
今回新たに追加するpnode-s1とpnode-s2でCESをイネーブルにする。
[root@snode-0 /]# mmchnode --ces-enable -N pnode-s1,pnode-s2
2020年 4月 1日 水曜日 15:07:47 JST: mmchnode: Processing node pnode-s1
The NFS service can not be enabled because it is not installed.
The file /usr/bin/gpfs.ganesha.nfsd was not found.
mmcesop: Command failed. Examine previous error messages to determine cause.
mmchnode: Node pnode-s1 cannot be added to CES.
2020年 4月 1日 水曜日 15:07:49 JST: mmchnode: Processing node pnode-s2
The NFS service can not be enabled because it is not installed.
The file /usr/bin/gpfs.ganesha.nfsd was not found.
mmcesop: Command failed. Examine previous error messages to determine cause.
mmchnode: Node pnode-s2 cannot be added to CES.
mmchnode: Command failed. Examine previous error messages to determine cause.
[root@snode-0 /]#
どうやらNFS serviceのインストールがCES Nodeになる必要条件のようだ。そこで、これら2つのノードにNFSをインストールする。[root@localhost rhel7]# rpm -ihv gpfs.nfs*
準備しています... ################################# [100%]
更新中 / インストール中...
1:gpfs.nfs-ganesha-2.7.5-ibm054.05.################################# [ 25%]
2:gpfs.nfs-ganesha-gpfs-2.7.5-ibm05################################# [ 50%]
3:gpfs.nfs-ganesha-utils-2.7.5-ibm0################################# [ 75%]
4:gpfs.nfs-ganesha-debuginfo-2.7.5-################################# [100%]
[root@localhost rhel7]#
再挑戦。
[root@snode-0 /]# mmchnode --ces-enable -N pnode-s1,pnode-s2
2020年 4月 1日 水曜日 16:45:30 JST: mmchnode: Processing node pnode-s1
2020年 4月 1日 水曜日 16:45:33 JST: mmchnode: Processing node pnode-s2
mmchnode: Propagating the cluster configuration data to all
affected nodes. This is an asynchronous process.
[root@snode-0 /]#
念の為の確認。
[root@snode-0 /]# mmces node list
Node Number Node Name Node Groups Node Flags
------------- ----------- ------------- ------------
3 pnode-1 nfsgroup1 none
4 pnode-2 nfsgroup1 none
5 pnode-s1 none
6 pnode-s2 none
[root@snode-0 /]#
今追加2つのCESノードで新たなグループ s3group1を構成する。
[root@snode-0 /]# mmchnode --ces-group s3group1 -N pnode-s1,pnode-s2
2020年 4月 1日 水曜日 16:57:37 JST: mmchnode: Processing node pnode-s1
2020年 4月 1日 水曜日 16:57:37 JST: mmchnode: Processing node pnode-s2
mmchnode: Propagating the cluster configuration data to all
affected nodes. This is an asynchronous process.
念の為の確認。
[root@snode-0 /]# mmces node list
Node Number Node Name Node Groups Node Flags
------------- ----------- ------------- ------------
3 pnode-1 nfsgroup1 none
4 pnode-2 nfsgroup1 none
5 pnode-s1 s3group1 none
6 pnode-s2 s3group1 none
[root@snode-0 /]#
このグループs3group1にCES IPとして192.168.1.108を割り当てる。
[root@snode-0 /]# mmces address add --ces-ip 192.168.1.108 --ces-group s3group1
念の為の確認。
[root@snode-0 /]# mmces address list
Address Node Ces Group Attributes
--------------- ---------- ----------- ------------
192.168.1.108 pnode-s1 s3group1 none
192.168.1.98 pnode-2 nfsgroup1 none
[root@snode-0 /]#
以下の内容でscale_object.repoを作り/etc/yum.repos.dに保存する。
[spectrum_scale]
name=spectrum_scale
baseurl=file:///usr/lpp/mmfs/5.0.4.3/object_rpms/rhel7
enabled=1
gpgcheck=0
OBJパッケージをインストールする。
[root@localhost rhel7]# yum -y install spectrum-scale-object
とりあえず以下の内容でパスワードファイルをつくり、ks-passwordとして/var/mmfs/ssl/keyServ/tmpに保存する。mmobjはパスワードファイルをこのディレクトリから読み出す。
ドキドキしながら以下を実行する。これはどれかのCES nodeで一回実行すればよい。CESノード全体が構成される。
ここで重要なのは既設のfilesetを指定していないところ。
[root@localhost rhel7]# mmobj swift base -g /gpfs/gpfsfs01 --cluster-hostname s3group1 --local-keystone --pwd-file ks-password -i 20000 --enable-s3
mmobj swift base: Validating execution environment.
mmobj swift base: Creating fileset gpfsfs01 object_fileset.
mmobj swift base: Configuring Keystone server in /gpfs/gpfsfs01/nfs/object/keystone.
mmobj swift base: Creating postgres database.
mmobj swift base: Setting cluster attribute object_database_node=192.168.1.108.
mmobj swift base: Validating Keystone environment.
mmobj swift base: Validating Swift values in Keystone.
mmobj swift base: Configuring Swift services.
mmobj swift base: Setting cluster attribute object_singleton_node=192.168.1.108.
mmobj swift base: Uploading configuration changes to the CCR.
mmobj swift base: Configuration complete.
[root@localhost rhel7]#
Defaultでobject_filesetがgpfs mount pointである/gpfs/gpfsfs01の下に作られた。
OBJをイネーブルしようとしたらエラーが出た。
[root@snode-0 /]# mmces service enable OBJ
pnode-s2: [E] The OBJ service cannot be enabled because it is not installed.
pnode-2: [E] The OBJ service cannot be enabled because it is not installed.
pnode-s2: The file /etc/swift/swift.conf was not found.
pnode-s2: mmcesop: Command failed. Examine previous error messages to determine cause.
mmdsh: pnode-s2 remote shell process had return code 1.
pnode-2: The file /etc/swift/swift.conf was not found.
pnode-2: mmcesop: Command failed. Examine previous error messages to determine cause.
mmdsh: pnode-2 remote shell process had return code 1.
pnode-1: [E] The OBJ service cannot be enabled because it is not installed.
pnode-1: The file /etc/swift/swift.conf was not found.
pnode-1: mmcesop: Command failed. Examine previous error messages to determine cause.
mmdsh: pnode-1 remote shell process had return code 1.
mmces service: Command failed. Examine previous error messages to determine cause.
pnode-s1: Keystone DB: service already stopped.
pnode-s1: Keystone: service already stopped.
pnode-s1: Swift: service already stopped.
mmces service enable: Command failed. Examine previous error messages to determine cause.
[root@snode-0 /]#
CESノード全てにOBJをインストールせねばならないようだ。で、以下を他のCESノードに対しても実行する。repoとパスワードファイルのコピーも忘れずに。
# yum -y install spectrum-scale-object
改めて実行。
[root@snode-0 /]# mmces service enable OBJ
mmchconfig: Command successfully completed
mmchconfig: Propagating the cluster configuration data to all
affected nodes. This is an asynchronous process.
pnode-s1: Keystone DB: service already stopped.
pnode-s2: Keystone DB: service already stopped.
pnode-s1: Keystone: service already stopped.
pnode-s2: Keystone: service already stopped.
pnode-2: Keystone DB: service already stopped.
pnode-2: Keystone: service already stopped.
pnode-1: Keystone DB: service already stopped.
pnode-1: Keystone: service already stopped.
pnode-s1: Swift: service already stopped.
pnode-s2: Swift: service already stopped.
pnode-2: Swift: service already stopped.
pnode-1: Swift: service already stopped.
pnode-s1: Version mismatch during upload of proxy-server.conf (1). Retrying.
pnode-s2: Version mismatch during upload of proxy-server.conf (2). Retrying.
pnode-s2: Version mismatch during upload of proxy-server.conf (3). Retrying.
pnode-2: Swift: service succesfully started.
pnode-s2: Swift: service succesfully started.
pnode-s1: Swift: service succesfully started.
pnode-1: Swift: service succesfully started.
pnode-s1: Keystone DB: service succesfully started.
pnode-s2: Keystone: service succesfully started.
pnode-s1: Keystone: service succesfully started.
pnode-1: Keystone: service succesfully started.
pnode-2: Keystone: service succesfully started.
[root@snode-0 /]#
上手くいったようだ。念の為の確認。これをNSDサーバーで実行したらエラーになった。CESノードで実行する必要があるようだ。
[root@snode-0 /]# mmces service list -v
Enabled services: NFS OBJ
mmces service: [E] The local node is not a CES node.
mmces service list: Command failed. Examine previous error messages to determine cause.
[root@snode-0 /]#
改めてCESノード上で確認。
[root@localhost rhel7]# mmces service list -v
Enabled services: NFS OBJ
NFS is running
OBJ is running
OBJ:openstack-swift-object-updater is running
OBJ:openstack-swift-object-expirer is running
OBJ:ibmobjectizer is not running
OBJ:openstack-swift-object is running
OBJ:openstack-swift-account is running
OBJ:openstack-swift-container is running
OBJ:memcached is running
OBJ:openstack-swift-proxy is running
OBJ:openstack-swift-object-replicator is running
OBJ:openstack-swift-account-reaper is running
OBJ:openstack-swift-account-auditor is running
OBJ:openstack-swift-container-auditor is running
OBJ:openstack-swift-container-updater is running
OBJ:openstack-swift-account-replicator is running
OBJ:openstack-swift-container-replicator is running
OBJ:openstack-swift-object-sof is not running
OBJ:postgresql-obj is running
OBJ:httpd (keystone) is running
[root@localhost rhel7]#
いい感じだ。念の為、更に確認。
[root@localhost rhel7]# mmces service list -a
Enabled services: NFS OBJ
pnode-2: NFS is running, OBJ is running
pnode-1: NFS is running, OBJ is running
pnode-s1: NFS is running, OBJ is running
pnode-s2: NFS is running, OBJ is running
[root@localhost rhel7]#
S3が動いているか確認。
[root@localhost rhel7]# mmobj s3 list
[I] The S3 API is enabled.
[root@localhost rhel7]#
とりあえず、S3が動いているようだ。
今日はここまで。次は実際にS3 APIでアクセスしてみるのだ。
« Scale 5.0.4.3にNFS CESをインストール | トップページ | Spectrum Scale - OpenStackへの扉 その1 »
「ソフトウエア導入」カテゴリの記事
- 非Windows11対応PCをWindows11にアップグレードする方法(2023.03.21)
- ThonnyでPicoのVersion upで SSL: CERTIFICATE_VERIFY_FAILED, certificate has expired にハマった件(2022.11.23)
- UI-VIEW32のオリジナルマップの作製(2022.11.22)
- APRS I-Gateの設置(2022.11.15)
- PCIe dual serial port adapterドライバーCH38XDRV.ZIPのWindows11へのインストール(2022.11.13)
« Scale 5.0.4.3にNFS CESをインストール | トップページ | Spectrum Scale - OpenStackへの扉 その1 »
コメント