대량 작업에 Oracle Cloud Infrastructure Object Storage Python 유틸리티 사용

소개

Oracle Cloud Infrastructure(OCI) Object Storage를 통해 고객은 모든 유형의 데이터를 기본 형식으로 안전하게 저장할 수 있습니다. OCI Object Storage는 내장된 리던던시를 통해 분석, 백업 또는 아카이브 목적으로 여러 데이터 소스를 통합하는 데 사용할 수 있으므로 확장성과 유연성이 필요한 최신 애플리케이션을 구축하는 데 이상적입니다.

OCI Object Storage는 클라우드 워크로드의 기초이며 탄력성, 확장성, 안정성 및 비용 효율성이 클라우드에서 구조화되지 않은 데이터를 위한 기본 스토리지로 만들었습니다. OCI Object Storage와 함께 사용할 수 있는 방대한 도구 에코시스템이 인기를 끌고 있습니다. 이 튜토리얼은 사용자가 사용 가능한 다양한 도구와 워크로드에서 사용하는 방법에 익숙해질 수있는 경로를 제공하는 것을 목표로하는 일련의 "전원 도구"의 첫 번째입니다.

이 첫 번째 자습서에서는 대량 작업을 위해 OCI Object Storage와 함께 Python SDK를 사용하여 GitHub에서 사용할 수 있는 예제 스크립트를 소개합니다.

대상

DevOps 엔지니어, 개발자, OCI Object Storage 관리자 및 사용자, IT 관리자, OCI 고급 사용자 및 애플리케이션 관리자.

목표

Python 스크립트 예제를 사용하여 대량 복원, 삭제, 복사, 이름 바꾸기 등 OCI Object Storage에서 대량 작업을 처리하고 객체 정보를 나열할 수 있습니다.

필수 조건

OCI 계정입니다.
.oci이라는 하위 디렉토리의 홈 디렉토리에 있는 작업 중인 구성 파일입니다. 자세한 내용은 구성 파일 설정을 참조하십시오.
OCI Object Storage 버킷에 액세스합니다.
OCI에서 OCI Object Storage를 사용할 수 있는 사용자 권한과 하나 이상의 버킷에서 객체를 관리할 수 있는 액세스 권한이 있습니다. 자세한 내용은 공통 정책 및 정책 참조를 참조하십시오.
실행 환경: OCI Cloud Shell에 대한 액세스, Linux OCI 인스턴스.
실행 환경에 설치된 Git 패키지와 git 명령에 익숙합니다.
Python 3이 설치되었습니다.
Python 및 Python 모듈 사용에 대한 기본적인 이해

작업 1: 환경 설정

OCI SDK(소프트웨어 개발 키트) 패키지를 설치합니다.
```
pip3 install --user oci
```

OCI SDK 저장소를 복제합니다.

$ git clone https://github.com/oracle/oci-python-sdk
Cloning into 'oci-python-sdk'...
remote: Enumerating objects: 113452, done.
remote: Counting objects: 100% (23584/23584), done.
remote: Compressing objects: 100% (3022/3022), done.
remote: Total 113452 (delta 20706), reused 23195 (delta 20521), pack-reused 89868
Receiving objects: 100% (113452/113452), 77.07 MiB | 12.78 MiB/s, done.
Resolving deltas: 100% (104373/104373), done.

참고: 이는 OCI Python SDK를 사용하는 예제 스크립트이며, 공식 Oracle 애플리케이션이 아니며 Oracle에서 유지 관리하거나 지원하지 않습니다. 이러한 예제는 예이므로 자체 소프트웨어 저장소에서 사용 사례에 맞게 검토, 업데이트 및 사용자 정의할 수 있습니다.

예제 디렉토리로 변경합니다. 홈 디렉토리에 설치된 경우 다음 명령을 실행합니다.
```
cd $HOME/oci-python-sdk/examples/object_storage
```

작업 2: 객체 대량 복원 스크립트 사용

대량 복원 스크립트는 병렬 스레드를 사용하여 소스 버킷의 아카이브된 객체를 복원합니다.

인수 없이 대량 복원 스크립트를 실행하여 필요한 인수를 확인합니다.

$ python object_storage_bulk_restore.py
Source bucket parameter is required !!!

usage: object_storage_bulk_restore.py [-h] [-t CONFIG_PROFILE] [-p PROXY]
                                      [-ip] [-dt] [-c CONFIG_FILE]
                                      [-sb SOURCE_BUCKET]
                                      [-sp SOURCE_PREFIX_INCLUDE]
                                      [-sr SOURCE_REGION]
                                      [-sn SOURCE_NAMESPACE]

optional arguments:
  -h, --help            show this help message and exit
  -t CONFIG_PROFILE     Config file section to use (tenancy profile)
  -p PROXY              Set Proxy (i.e. www-proxy-server.com:80)
  -ip                   Use Instance Principals for Authentication
  -dt                   Use Delegation Token for Authentication
  -c CONFIG_FILE        Config File (default=~/.oci/config)
  -sb SOURCE_BUCKET     Source Bucket Name
  -sp SOURCE_PREFIX_INCLUDE
                        Source Prefix Include
  -sr SOURCE_REGION     Source Region
  -sn SOURCE_NAMESPACE  Source Namespace (Default current connection)

주: 이 자습서에서는 모든 스크립트에 대해 구성 파일을 사용합니다. 스크립트가 스크립트 인수로 지정되지 않은 경우 스크립트는 기본적으로 구성 파일에 나열된 영역으로 설정됩니다.

접두어 인수를 사용하지 않고 스크립트를 실행합니다. 그러면 모든 아카이브 객체가 복원됩니다.

$ python object_storage_bulk_restore.py -c $HOME/.oci/config -sb MyBucket

Connecting to Object Storage Service...
Succeed.

##########################################################################################
#                          Running Object Storage Bulk Restore                           #
##########################################################################################
Written by Tim S and Adi Z, July 2020
Starts at             : 2024-05-30 16:44:18
Command Line          : -c /home/opc/.oci/config -sb MyBucket
Source Namespace      : namespace
Source Bucket         : MyBucket
Source Prefix Include :

##########################################################################################
#                                    Start Processing                                    #
##########################################################################################
16:44:18 - Creating 40 workers.
16:44:18 - Getting list of objects from source source_bucket (MyBucket). Restores will start immediately.
16:44:19 - Enqueued 9086 objects to be restored
16:44:19 - Waiting 60 seconds before checking status.
16:45:19 - Restoration of all objects has been requested.

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2024-05-30 16:45:19

주: 복원된 객체는 즉시 사용할 수 없습니다. 복원은 아카이브 스토리지 복원 요청이 수행된 시간부터 첫번째 데이터 바이트가 검색되는 시간까지 최대 1시간 정도 걸립니다.

접두어 인수가 있는 스크립트를 실행합니다

$ python object_storage_bulk_restore.py -c $HOME/.oci/config -sb MyBucket -sp folder1

Connecting to Object Storage Service...
Succeed.

##########################################################################################
#                          Running Object Storage Bulk Restore                           #
##########################################################################################
Written by Tim S and Adi Z, July 2020
Starts at             : 2024-05-30 17:00:09
Command Line          : -c /home/opc/.oci/config -sb MyBucket -sp folder1
Source Namespace      : namespace
Source Bucket         : MyBucket
Source Prefix Include : folder1

##########################################################################################
#                                    Start Processing                                    #
##########################################################################################
17:00:09 - Creating 40 workers.
17:00:09 - Getting list of objects from source source_bucket (MyBucket). Restores will start immediately.
17:00:10 - Enqueued 1096 objects to be restored
17:00:10 - Waiting 60 seconds before checking status.
17:01:10 - Restoration of all objects has been requested.

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2024-05-30 17:01:10

주: 이름이 지정된 접두어/폴더 및 하위 접두어/하위 폴더에 있는 모든 아카이브된 객체가 복원됩니다.

작업 3: 객체 대량 삭제 스크립트 사용

객체 대량 삭제 스크립트는 필터링 옵션을 사용하여 버킷에서 객체를 삭제합니다.

인수 없이 스크립트를 실행하여 사용량을 확인합니다.

$ python object_storage_bulk_delete.py
Source bucket parameter is required !!!

usage: object_storage_bulk_delete.py [-h] [-t CONFIG_PROFILE] [-p PROXY] [-ip]
                                    [-dt] [-c CONFIG_FILE]
                                    [-sb SOURCE_BUCKET] [-sp SOURCE_PREFIX]
                                    [-se SOURCE_PREFIX_EXCLUDE]
                                    [-exclude_dirs] [-sn SOURCE_NAMESPACE]
                                    [-sr SOURCE_REGION]

optional arguments:
  -h, --help            show this help message and exit
  -t CONFIG_PROFILE     Config file section to use (tenancy profile)
  -p PROXY              Set Proxy (i.e. www-proxy-server.com:80)
  -ip                   Use Instance Principals for Authentication
  -dt                   Use Delegation Token for Authentication
  -c CONFIG_FILE        Config File (default=~/.oci/config)
  -sb SOURCE_BUCKET     Source Bucket Name
  -sp SOURCE_PREFIX     Source Prefix Include
  -se SOURCE_PREFIX_EXCLUDE
                        Source Prefix Exclude
  -exclude_dirs         Exclude Directories
  -sn SOURCE_NAMESPACE  Source Namespace (Default current connection)
  -sr SOURCE_REGION     Source Region

참고: 포함 또는 제외 접두어 없이 스크립트를 실행하면 버킷의 모든 항목이 삭제됩니다.

대량 삭제 스크립트를 실행하고 포함 접두어를 설정하고 출력을 검토한 다음 y/N 프롬프트에 응답합니다.

주: 접두어/폴더 및 하위 접두어/하위 폴더의 모든 객체가 삭제됩니다. 명령을 실행할 때는 주의해야 합니다. 일치 항목을 찾을 수 없는 경우 버킷의 모든 항목이 삭제됩니다.

$ python object_storage_bulk_delete.py  -c $HOME/.oci/config -sb MyBucket -sp folder1

Connecting to Object Storage Service...
Succeed.

##########################################################################################
#                           Running Object Storage Bulk Delete                           #
##########################################################################################
Written by Adi Zohar, July 2020
Starts at             : 2024-05-30 18:51:32
Command Line          : -c /home/opc/.oci/config -sb MyBucket -sp folder1
Source Namespace      : namespace
Source Bucket         : MyBucket
Source Prefix Include : folder1
Source Prefix Exclude :
Source Region         : us-ashburn-1

Are you sure you want to continue deleting ? [y/N]: y

##########################################################################################
#                                    Start Processing                                    #
##########################################################################################
18:51:38 - Creating 40 workers.
18:51:38 - Getting list of objects from source source_bucket (MyBucket). delete will start immediately.
18:51:38 - Enqueued 700 objects to be deleted
18:51:38 - Waiting 60 seconds before checking status.
18:52:38 - deletion of all objects has been requested.

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2024-05-30 18:52:38

제외 접두어를 사용하여 대량 삭제 스크립트를 실행합니다.

$ python object_storage_bulk_delete.py  -c $HOME/.oci/config -sb MyBucket -se folder1

Connecting to Object Storage Service...
Succeed.

##########################################################################################
#                           Running Object Storage Bulk Delete                           #
##########################################################################################
Written by Adi Zohar, July 2020
Starts at             : 2024-05-30 19:02:53
Command Line          : -c /home/opc/.oci/config -sb MyBucket -se folder1
Source Namespace      : namespace
Source Bucket         : MyBucket
Source Prefix Include :
Source Prefix Exclude : folder1
Source Region         : us-ashburn-1

Are you sure you want to continue deleting ? [y/N]: y

##########################################################################################
#                                    Start Processing                                    #
##########################################################################################
19:02:58 - Creating 40 workers.
19:02:58 - Getting list of objects from source source_bucket (MyBucket). delete will start immediately.
19:02:58 - Enqueued 60 objects to be deleted
19:02:58 - Waiting 60 seconds before checking status.
19:03:58 - deletion of all objects has been requested.

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2024-05-30 19:03:58

참고: 제외 접두어/폴더에 없는 버킷의 모든 객체가 삭제됩니다. 이 스크립트를 실행할 때는 주의하십시오. 일치 항목을 찾을 수 없으면 버킷의 모든 객체가 삭제됩니다.

작업 4: 객체 대량 이름 바꾸기 스크립트 사용

대량 이름 바꾸기 스크립트를 사용하여 객체의 이름을 바꾸거나 객체를 다른 폴더/접두어로 이동할 수 있습니다. 이름 바꾸기 스크립트는 단순히 객체의 이름 메타 데이터를 변경하고, 객체의 복사본이 아니며, 이름 바꾸기일 뿐이라는 점을 이해하는 것이 중요합니다.

필수 매개변수를 보려면 인수 없이 객체 대량 이름 바꾸기 스크립트를 실행하십시오.

$ python object_storage_bulk_rename.py
Source bucket parameter is required !!!

usage: object_storage_bulk_rename.py [-h] [-t CONFIG_PROFILE] [-p PROXY] [-ip]
                                    [-dt] [-c CONFIG_FILE]
                                    [-sb SOURCE_BUCKET]
                                    [-sp SOURCE_PREFIX_INCLUDE]
                                    [-sr SOURCE_REGION]
                                    [-sn SOURCE_NAMESPACE]
                                    [-textrem TEXT_REMOVE]
                                    [-textadd TEXT_APPEND]

optional arguments:
  -h, --help            show this help message and exit
  -t CONFIG_PROFILE     Config file section to use (tenancy profile)
  -p PROXY              Set Proxy (i.e. www-proxy-server.com:80)
  -ip                   Use Instance Principals for Authentication
  -dt                   Use Delegation Token for Authentication
  -c CONFIG_FILE        Config File (default=~/.oci/config)
  -sb SOURCE_BUCKET     Source Bucket Name
  -sp SOURCE_PREFIX_INCLUDE
                        Source Prefix Include
  -sr SOURCE_REGION     Source Region
  -sn SOURCE_NAMESPACE  Source Namespace (Default current connection)
  -textrem TEXT_REMOVE  text remove prefix (can be used to remove folder)
  -textadd TEXT_APPEND  text append prefix (can be used to add folder)

os 객체 목록을 실행하여 버킷에 현재 객체를 표시합니다.

기존 객체:

$ oci os object list --bucket-name MyBucket --query "data[].name"
[
  "file_1.txt",
  "file_10.txt",
  "file_2.txt",
  "file_3.txt",
  "file_4.txt",
  "file_5.txt",
  "file_6.txt",
  "file_7.txt",
  "file_8.txt",
  "file_9.txt",
  "hello_world"
]

객체 대량 이름 바꾸기 스크립트를 실행하여 객체 이름을 시작 부분에 file로 변경하고 object로 바꿉니다.

$ python object_storage_bulk_rename.py -c $HOME/.oci/config -sb MyBucket -textrem file -textadd object

Connecting to Object Storage Service...
Succeed.

##########################################################################################
#                           Running Object Storage Bulk Rename                           #
##########################################################################################
Written by Adi Z, March 2021
Starts at             : 2024-05-31 23:09:13
Command Line          : -c /home/opc/.oci/config -sb MyBucket -textrem file -textadd object
Source Namespace      : namespace
Source Bucket         : MyBucket
Source Prefix Include : file
Text Remove Prefix    : file
Text Append Prefix    : object

##########################################################################################
#                                    Start Processing                                    #
##########################################################################################
23:09:13 - Creating 40 workers.
23:09:13 - Getting list of objects from source source_bucket (MyBucket). Rename will start immediately.
23:09:13 - Enqueued 10 objects to be Renamed
23:09:13 - Waiting 60 seconds before checking status.
23:10:13 - Rename of all objects has been requested.

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2024-05-31 23:10:13

os 객체 목록을 실행하여 버킷에 현재 객체를 표시하고 이름 변경 사항을 확인합니다.

$ oci os object list --bucket-name MyBucket --query "data[].name"
[
  "hello_world",
  "object_1.txt",
  "object_10.txt",
  "object_2.txt",
  "object_3.txt",
  "object_4.txt",
  "object_5.txt",
  "object_6.txt",
  "object_7.txt",
  "object_8.txt",
  "object_9.txt"
]

이름에 object 문자열이 있는 모든 객체를 dir1 디렉토리로 이동합니다.

$ python object_storage_bulk_rename.py -c $HOME/.oci/config -sb MyBucket -sp object -textadd dir1/

Connecting to Object Storage Service...
Succeed.

##########################################################################################
#                           Running Object Storage Bulk Rename                           #
##########################################################################################
Written by Adi Z, March 2021
Starts at             : 2024-05-31 23:16:14
Command Line          : -c /home/opc/.oci/config -sb MyBucket -sp object -textadd dir1/
Source Namespace      : namespace
Source Bucket         : MyBucket
Source Prefix Include : object
Text Remove Prefix    :
Text Append Prefix    : dir1/

##########################################################################################
#                                    Start Processing                                    #
##########################################################################################
23:16:14 - Creating 40 workers.
23:16:14 - Getting list of objects from source source_bucket (MyBucket). Rename will start immediately.
23:16:15 - Enqueued 10 objects to be Renamed
23:16:15 - Waiting 60 seconds before checking status.
23:17:15 - Rename of all objects has been requested.

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2024-05-31 23:17:15

os 객체 목록을 실행하여 버킷의 현재 객체를 다시 표시하고 이름이 dir1인 폴더/접두어로 이동된 객체를 확인합니다.

$ oci os object list --bucket-name MyBucket --query "data[].name"
[
  "dir1/object_1.txt",
  "dir1/object_10.txt",
  "dir1/object_2.txt",
  "dir1/object_3.txt",
  "dir1/object_4.txt",
  "dir1/object_5.txt",
  "dir1/object_6.txt",
  "dir1/object_7.txt",
  "dir1/object_8.txt",
  "dir1/object_9.txt",
  "hello_world"
]

작업 5: 객체 대량 복사 스크립트 사용

대량 복사 스크립트는 버킷 간에 객체를 복사하며, 다른 지역 또는 다른 테넌시의 버킷 간에 객체를 복사하는 데 사용할 수 있습니다. 이 사용지침서에서는 리전 간 동일한 테넌시의 버킷 간 복사본, us-ashburn-1의 버킷 하나와 us-phoenix-1의 다른 버킷 간 복사본을 보여줍니다. 테넌시 전체에서 스크립트를 사용하려면 테넌시 전체에서 오브젝트 스토리지 리소스 액세스를 참조하십시오.

주: 버킷 간에 객체를 대량 복사할 수 있는 보다 강력한 도구(예: Rclone 또는 Obect Storage Replication)가 있습니다.

필수 매개변수를 이해하기 위해 인수 없이 객체 대량 복사 스크립트를 실행합니다.

$ python object_storage_bulk_copy.py
usage: object_storage_bulk_copy.py [-h] [-t CONFIG_PROFILE] [-p PROXY] [-ip]
                                  [-dt] [-c CONFIG_FILE] [-sb SOURCE_BUCKET]
                                  [-sr SOURCE_REGION] [-sn SOURCE_NAMESPACE]
                                  [-sp SOURCE_PREFIX_INCLUDE]
                                  [-se SOURCE_PREFIX_EXCLUDE]
                                  [-db DESTINATION_BUCKET]
                                  [-dr DESTINATION_REGION]
                                  [-dn DESTINATION_NAMESPACE] [-ig]

optional arguments:
  -h, --help            show this help message and exit
  -t CONFIG_PROFILE     Config file section to use (tenancy profile)
  -p PROXY              Set Proxy (i.e. www-proxy-server.com:80)
  -ip                   Use Instance Principals for Authentication
  -dt                   Use Delegation Token for Authentication
  -c CONFIG_FILE        Config File (default=~/.oci/config)
  -sb SOURCE_BUCKET     Source Bucket Name
  -sr SOURCE_REGION     Source Region (Default current connection)
  -sn SOURCE_NAMESPACE  Source Namespace (Default current connection)
  -sp SOURCE_PREFIX_INCLUDE
                        Source Prefix Include
  -se SOURCE_PREFIX_EXCLUDE
                        Source Prefix Exclude
  -db DESTINATION_BUCKET
                        Destination Bucket Name
  -dr DESTINATION_REGION
                        Destination Region
  -dn DESTINATION_NAMESPACE
                        Destination Namespace (Default current connection)
  -ig                   Ignore Check if files exist at Destination

객체 대량 객체 복사 명령을 실행하여 폴더/접두어의 객체를 다른 영역의 버킷에 복사합니다.

주: 이 예에서는 대상 버킷이 비어 있습니다.

$ python object_storage_bulk_copy.py -c $HOME/.oci/config -sb MySourceBucket -db MyDestBucket -dr us-phoenix-1

##########################################################################################
#                              Connecting to Object Storage                              #
##########################################################################################

Connecting to Object Storage Service for source region - us-ashburn-1
Succeed.

Connecting to Object Storage Service for destination region - us-phoenix-1
Succeed.

##########################################################################################
#                            Running Object Storage Bulk Copy                            #
##########################################################################################
Written by Tim S and Adi Z, July 2020
Starts at        : 2024-05-31 19:38:52
Command Line     : -c /home/opc/.oci/config -sb MySourceBucket -db MyDestBucket
Source Namespace : namespace
Source Region    : us-ashburn-1
Source Bucket    : MySourceBucket
Source Prefix    :
Dest   Namespace : namespace
Dest   Region    : us-phoenix-1
Dest   Bucket    : MyDestBucket
State  File      : MySourceBucket.bulk.wrk

##########################################################################################
#                                    Start Processing                                    #
##########################################################################################
19:38:52 - Creating 50 copy request workers.
19:38:52 - Creating 50 status workers.
19:38:52 - Loading list of objects from destination bucket (MyDestBucket) to ignore exiting files.
19:38:52 -    Loaded 0 files.
19:38:52 - Getting list of objects from source bucket (MySourceBucket). Copies will start immediately.
19:38:52 -    Enqueued 5 objects to be copied

[...snip...]

##########################################################################################
#                          Finish queuing files, start checking                          #
##########################################################################################
19:38:52 - Waiting 60 seconds before checking status.
19:39:52 - Determining copy status
19:39:52 -    KNOWN: 1, REQUESTED: 0, COMPLETED: 4, FAILED: 0, CANCELED: 0

작업 6: 객체 스토리지 목록 폴더 스크립트 사용

오브젝트 스토리지 목록 폴더 스크립트에는 버킷의 각 폴더/접두어, 각 폴더의 모든 오브젝트(킬로바이트)의 총 크기 및 각 폴더의 오브젝트 수가 나열됩니다. 접두어/폴더별로 필터링할 수 있는 옵션이 있습니다.

명령행 옵션을 이해하기 위해 인수 없이 객체 목록 폴더 스크립트를 실행합니다.

$ python  object_storage_list_folders.py
Source bucket parameter is required !!!

usage: object_storage_list_folders.py [-h] [-t CONFIG_PROFILE] [-p PROXY]
                                      [-ip] [-dt] [-c CONFIG_FILE]
                                      [-sb SOURCE_BUCKET] [-sp SOURCE_PREFIX]
                                      [-sr SOURCE_REGION]
                                      [-sn SOURCE_NAMESPACE] [-f FILE]

optional arguments:
  -h, --help            show this help message and exit
  -t CONFIG_PROFILE     Config file section to use (tenancy profile)
  -p PROXY              Set Proxy (i.e. www-proxy-server.com:80)
  -ip                   Use Instance Principals for Authentication
  -dt                   Use Delegation Token for Authentication
  -c CONFIG_FILE        Config File (default=~/.oci/config)
  -sb SOURCE_BUCKET     Source Bucket Name
  -sp SOURCE_PREFIX     Source Prefix Include
  -sr SOURCE_REGION     Source Region
  -sn SOURCE_NAMESPACE  Source Namespace (Default current connection)
  -f FILE               Output to file (as csv)

전체 버킷에 대해 객체 목록 폴더 스크립트를 실행합니다.

$ python  object_storage_list_folders.py -c $HOME/.oci/config -sb MyBucket

Connecting to Object Storage Service...
Success.

##########################################################################################
#                               Running List/Count Objects                               #
##########################################################################################
Written By Adi Zohar, June 2020
Starts at           :2024-05-30 21:25:06
Command Line        : -c /home/opc/.oci/config -sb MyBucket
Source Namespace    : namespace
Source Bucket       : MyBucket
Source Prefix       :

##########################################################################################
#                                  Start Processing...                                   #
##########################################################################################
Folder Name                                        |  Size (KB) |   Count
--------------------------------------------------------------------------------
dir_1/                                             | 10240000.00 |      10
dir_10/                                            | 10240000.00 |      10
dir_2/                                             | 10240000.00 |      10
dir_3/                                             | 10240000.00 |      10
dir_4/                                             | 10240000.00 |      10
dir_5/                                             | 10240000.00 |      10
dir_6/                                             | 10240000.00 |      10
dir_7/                                             | 10240000.00 |      10
dir_8/                                             | 10240000.00 |      10
dir_9/                                             | 10240000.00 |      10

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2024-05-30 21:25:06
Total Files  :                  103
Total Size   :      137,069,854,720

특정 폴더/접두어에 대해 객체 목록 폴더 스크립트를 실행합니다.

$ python  object_storage_list_folders.py -c $HOME/.oci/config -sb  MyBucket -sp dir_1/

Connecting to Object Storage Service...
Success.

##########################################################################################
#                               Running List/Count Objects                               #
##########################################################################################
Written By Adi Zohar, June 2020
Starts at           :2024-05-30 21:27:41
Command Line        : -c /home/opc/.oci/config -sb MyBucket -sp dir_1/
Source Namespace    : namespace
Source Bucket       : MyBucket
Source Prefix       : dir_1/

##########################################################################################
#                                  Start Processing...                                   #
##########################################################################################
Folder Name                                        |  Size (KB) |   Count
--------------------------------------------------------------------------------
dir_1/                                             | 10240000.00 |      10

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2024-05-30 21:27:42
Total Files  :                   10
Total Size   :       10,485,760,000

작업 7: 객체 스토리지 목록 객체 스크립트 사용

객체 목록 스크립트는 버킷에 있는 각 객체, 각 객체의 크기(바이트), 모든 객체의 총 크기 및 모든 객체의 수를 나열합니다. 접두어/폴더별로 필터링할 수 있는 옵션이 있습니다. 객체 목록 스크립트는 문자열을 기반으로 버킷 또는 접두어에서 객체를 검색하는 데 사용할 수도 있습니다.

전체 버킷에 대해 객체 목록 스크립트를 실행합니다.

$ python object_storage_list_objects.py -c  $HOME/.oci/config -sb MyBucket

Connecting to Object Storage Service...
Success.

##########################################################################################
#                               Running List/Count Objects                               #
##########################################################################################
Written By Adi Zohar, June 2020
Starts at           :2024-05-31 18:25:16
Command Line        : -c /home/opc/.oci/config -sb MyBucket
Source Namespace    : namespace
Source Bucket       : MyBucket
Source Prefix       :
Source Pre-Exclude  :

##########################################################################################
#                                  Start Processing...                                   #
##########################################################################################
      10,737,418,240 | C 2024-01-30 23:58 | U 2024-01-30 23:58 | Standard          | 10GB-Testfile
      10,737,418,240 | C 2024-01-25 19:41 | U 2024-01-25 19:41 | Standard          | 10GB-file
      10,737,418,240 | C 2024-01-24 22:13 | U 2024-01-24 22:13 | Standard          | 10GB_file
      1,048,576,000 | C 2024-01-24 22:13 | U 2024-01-24 22:13 | Standard          | dir_1/file_1
      1,048,576,000 | C 2024-01-24 22:14 | U 2024-01-24 22:14 | Standard          | dir_1/file_10

[...snip...]

      1,048,576,000 | C 2024-01-24 23:10 | U 2024-01-24 23:10 | Standard          | dir_9/file_8
      1,048,576,000 | C 2024-01-24 23:10 | U 2024-01-24 23:10 | Standard          | dir_9/file_9

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2024-05-31 18:25:16
Total Files  :                  103
Total Size   :      137,069,854,720

목록 객체 스크립트를 실행하고 폴더/접두어별로 필터링합니다.

$ python object_storage_list_objects.py -c  $HOME/.oci/config -sb MyBucket -sp dir_1/

Connecting to Object Storage Service...
Success.

##########################################################################################
#                               Running List/Count Objects                               #
##########################################################################################
Written By Adi Zohar, June 2020
Starts at           :2024-05-31 18:49:49
Command Line        : -c /home/opc/.oci/config -sb MyBucket -sp dir_1/
Source Namespace    : namespace
Source Bucket       : MyBucket
Source Prefix       : dir_1/
Source Pre-Exclude  :

##########################################################################################
#                                  Start Processing...                                   #
##########################################################################################
      1,048,576,000 | C 2024-01-24 22:13 | U 2024-01-24 22:13 | Standard          | dir_1/file_1
      1,048,576,000 | C 2024-01-24 22:14 | U 2024-01-24 22:14 | Standard          | dir_1/file_10
      1,048,576,000 | C 2024-01-24 22:15 | U 2024-01-24 22:15 | Standard          | dir_1/file_2
      1,048,576,000 | C 2024-01-24 22:16 | U 2024-01-24 22:16 | Standard          | dir_1/file_3
      1,048,576,000 | C 2024-01-24 22:17 | U 2024-01-24 22:17 | Standard          | dir_1/file_4
      1,048,576,000 | C 2024-01-24 22:17 | U 2024-01-24 22:17 | Standard          | dir_1/file_5
      1,048,576,000 | C 2024-01-24 22:17 | U 2024-01-24 22:17 | Standard          | dir_1/file_6
      1,048,576,000 | C 2024-01-24 22:18 | U 2024-01-24 22:18 | Standard          | dir_1/file_7
      1,048,576,000 | C 2024-01-24 22:19 | U 2024-01-24 22:19 | Standard          | dir_1/file_8
      1,048,576,000 | C 2024-01-24 22:20 | U 2024-01-24 22:20 | Standard          | dir_1/file_9

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2024-05-31 18:49:49
Total Files  :                   10
Total Size   :       10,485,760,000

목록 객체 스크립트를 실행하고 접두어/폴더에 있는 객체의 시작 부분에 있는 문자열의 폴더/접두어를 필터링합니다.

$ python object_storage_list_objects.py -sb MyBucket -sp dir2/foo

Connecting to Object Storage Service...
Success.

##########################################################################################
#                               Running List/Count Objects                               #
##########################################################################################
Written By Adi Zohar, June 2020
Starts at           :2025-09-19 23:09:47
Command Line        : -sb ash-30M -sp dir2/foo
Source Namespace    : ocitsammut
Source Bucket       : MyBucket
Source Prefix       : dir2/foo
Source Pre-Exclude  : 

##########################################################################################
#                                  Start Processing...                                   #
##########################################################################################
                   4 | C 2025-09-19 21:34 | U 2025-09-19 21:34 | Standard          | dirr2/foo.txt
                   4 | C 2025-09-19 21:34 | U 2025-09-19 21:34 | Standard          | dir2/foo1.txt
                   4 | C 2025-09-19 21:34 | U 2025-09-19 21:34 | Standard          | dir2/foo2.txt

##########################################################################################
#                                       Completed                                        #
##########################################################################################
Completed at :  2025-09-19 23:09:47
Total Files  :                    3
Total Size   :                   12

승인

작성자 - Melinda Centeno(수석 수석 제품 관리자)
제공업체 - Adi Zohar(Master Principal Cloud Architect), Tim Sammut(기술 담당자 상담 멤버)

추가 학습 자원

docs.oracle.com/learn에서 다른 랩을 탐색하거나 Oracle Learning YouTube 채널에서 더 많은 무료 학습 콘텐츠에 액세스하세요. 또한 education.oracle.com/learning-explorer를 방문하여 Oracle Learning Explorer가 되십시오.

제품 설명서는 Oracle Help Center를 참조하십시오.

제목 및 저작권 정보

Use Oracle Cloud Infrastructure Object Storage Python Utilities for Bulk Operations

G10216-03

Oracle and/or its affiliates.