Domino Consulting - HCL Domino Administrator 11.0.1 Help

ADMINISTERING

Reverting use of DAOS tier 2 storage

HCL Domino does not currently provide an automated way to move DAOS objects from tier 2 storage to tier 1 storage. The following steps describe how to revert manually.

About this task

The overall steps are as follows. See the detailed steps for more information.

1. Disable DAOS tier 2.

2. Verify that there are objects in tier 2.

3. Copy all tier 2 objects to tier 1 storage.

4. Perform a quick resync.

5. Verify that DAOS no longer has tier 2 references.

6. Delete tier 2 objects.

7. Perform a full resync.

Procedure

1. Disable DAOS tier 2 storage:

On the DAOS tab of the Server document, disable the

DAOS Tier 2

field.

b. Restart the Domino server on which you made the change:

restart server

When the server comes back up, enter the following command at the server console:

tell daosmgr status

Verify that the output says DAOS Tier2 is Disabled.

Note: Although tier 2 is disabled, existing objects that are stored there can still be downloaded by Domino, because its object index is still aware that those objects are stored in tier 2. No additional objects will be pushed to tier 2 at this point, though.

2. Verify that there are DAOS objects in tier 2 by issuing the following command at the server console: .

tell daosmgr objectinfo summary

If the output indicates that there are no tier 2 objects, skip the remaining steps in this procedure. The following example output indicates that there are 32 tier 2 objects:

[1998:0002-082C] DAOS TIERED STORAGE SUMMARY[1998:0002-082C] TIER 1 STORAGE:[1998:0002-082C] OBJECTS - 0[1998:0002-082C] MB - 0 MB[1998:0002-082C] TIER 2 STORAGE:[1998:0002-082C] OBJECTS - 32[1998:0002-082C] MB - 70 MB[1998:0002-082C] TOTAL STORAGE :[1998:0002-082C] OBJECTS - 32[1998:0002-082C] MB - 70 MB[1998:0002-082C] AVERAGE AGE - 1521 days

3. Copy all tier 2 objects to tier 1 storage. These steps assume the use of AWS CLI (command line interface), which must be installed and configured. But you can copy using any tool that works with your S3 backend.

Determine the profile name. The AWS command line interface (CLI) typically get the S3 credentials from a file named

"credentials"

in the

".aws"

subdirectory under your user home directory. The credentials file can have multiple credentials; each one has a profile name in brackets, such as

[dominocos]

, an

aws_access_key_id

, and an

aws_secret_access_key

. If you are using the [default] credential, you needn't worry about the profile name. Otherwise you will need to specify the correct profile name in your S3 commands.

b. To gather additional required information, issue the following command at the server console:

tell daosmgr status

Gather the following information from the command output:

Base Tier 1 Storage Path: Shown as baseStoragePath in the output.
S3 Endpoint: Shown as DAOS Tier2 Endpoint in the output. If using AWS S3, the endpoint may not be required. If using MinIO, use the endpoint of your MinIO server, for example:https://minio.myserver.renovations.com.
S3 Bucket: Shown as DAOS Tier2 Bucket in the output.
S3 Server ID: Shown as DAOS Tier2 Server ID in the output.

(Optional) To list all of your tier 2 objects, compose a command of the following form. Be aware that the command may take an extended period of time to complete.

aws s3 [--profile <profile-name>] [--endpoint-url <s3-endpoint>] ls

s3://<s3-bucket>/obj/<s3-server-id>/<s3-server-id>/ --recursive

Note: The case must match exactly for the source path, and on non-Windows systems, for the target path.

For example:

aws s3 --profile dominocos ls s3://aws-hcl-dominocos/obj/96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ --recursive

Example output that lists two objects. You are likely to have more.

2019-10-09 15:55:02 1777421 obj/96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 040E4598A2A287490B8DD7784FEB7150B876313E001B1D64.nlo2019-10-09 15:55:01 1771669 obj/96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 0E597BF9C316422FCA869567755CD882C8DCC469001B06EC.nlo

Copy your tier 2 objects to tier 1. You can choose any DAOS subcontainer as the destination directory on your Domino server. In this example we assume 0001. Execute a command of the following form. The command may take an extensive period of time to complete.

aws s3 [--profile <profile-name>] [--endpoint-url <s3-endpoint>] cp s3://<s3-bucket>/obj/<s3-server-id>/

<s3-server-id>/<base-tier1-storage-path>/0001 --recursive

For example:

aws s3 --profile dominocos cp s3://aws-hcl-dominocos/obj/96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ C:\Domino\MyServer\Data\DAOS\0001 --recursive

Example console output:

download: s3://aws-hcl-dominocos/obj/96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 1FAE990C593D66B44923C1C0E41740F9628DDD170000D902.nlo to .\ 1FAE990C593D66B44923C1C0E41740F9628DDD170000D902.nlodownload:s3://aws-hcl-dominocos/obj/96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 11FCA4B81120D5650D67C9427C80D6B8256AFCF30000D902.nlo to .\ 11FCA4B81120D5650D67C9427C80D6B8256AFCF30000D902.nlo

4. Perform a quick resync by issuing the following command at the server console:

tell daosmgr resync quick force

This causes DAOS to refresh its index of objects in both tier 1 and tier 2 but does not do a full resync. If you issue a tell daosmgr status command, the results showcatalogState = RESYNCING. DAOS continues to operate in this state, but doesn't prune any deleted objects.

5. verify that there are no longer tier 2 objects by issuing the following command at the server console:

tell daosmgr objectinfo summary

The output should indicate 0 tier 2 objects, for example:

[1D1C:0002-6170] DAOS TIERED STORAGE SUMMARY[1D1C:0002-6170] TIER 1 STORAGE:[1D1C:0002-6170] OBJECTS - 32[1D1C:0002-6170] MB - 70 MB[1D1C:0002-6170] TIER 2 STORAGE:[1D1C:0002-6170] OBJECTS - 0[1D1C:0002-6170] MB - 0 MB[1D1C:0002-6170] TOTAL STORAGE :[1D1C:0002-6170] OBJECTS - 32[1D1C:0002-6170] MB - 70 MB[1D1C:0002-6170] AVERAGE AGE - 1521 days

6. Delete tier 2 objects from S3 by issuing a command in the following form. This command is the same as the command in Step 3c to list tier 2 objects except you replace the lsflag with rm:

aws s3 [--profile <profile-name>] [--endpoint-url <s3-endpoint>] rm

s3://<s3-bucket>/obj/<s3-server-id>/<s3-server-id>/ --recursive

For example:

aws s3 --profile dominocos rm s3://aws-hcl-dominocos/obj/96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ --recursive

Example console output:

delete:s3://aws-hcl-dominocos/obj/96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 040E4598A2A287490B8DD7784FEB7150B876313E001B1D64.nlodelete:s3://aws-hcl-dominocos/obj/96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 96A6456FB0F12AE91178C9A8AE0DAB7DB6F1130D/ 11FCA4B81120D5650D67C9427C80D6B8256AFCF30000D902.nlo

7. Perform a full DAOS resync in either of the following ways.

If there is a Domino directory Program document scheduled to run the tell daosmgr resync command, wait for it to run.
Otherwise, issue the command tell daosmgr resync force at the server console. A full resync may take an extended period of time to complete, so running it during non-business hours is preferable.

Note:

The

tell daosmgr resync quick force

command that you issued in Step 4 assured that DAOS has an index of all existing objects. But the command does not assure that all the reference counts on the objects are precise. Precise reference counts requires a full resync, which involves rebuilding the object index from scratch by iterating over all attachment references in all DAOS-enabled databases.

Help on Help

All Help Contents

Help on Help

All Help Contents