728x90
반응형
s3cmd란?
S3cmd 는 데이터 S3 호환 개체 저장소를 업로드, 검색 및 관리하기 위한 무료 오픈 소스 명령줄 도구 및 클라이언트이다. 명령줄 프로그램에 익숙하지만 초보자도 빠르게 배울 수 있을 만큼 간단한 고급 사용자를 위한 강력한 도구이다.
기본환경
- python 3.x 이상
s3cmd 설치방법
1. s3cmd 설치
python만 설치되어 있다면 설치방법은 매우 간단하다.
hadoop@hadoop-001:~$ pip install s3cmd
Collecting s3cmd
Downloading s3cmd-2.1.0-py2.py3-none-any.whl (145 kB)
|████████████████████████████████| 145 kB 199 kB/s
Collecting python-dateutil
Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting python-magic
Downloading python_magic-0.4.22-py2.py3-none-any.whl (12 kB)
Collecting six>=1.5
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six, python-magic, python-dateutil, s3cmd
Successfully installed python-dateutil-2.8.1 python-magic-0.4.22 s3cmd-2.1.0 six-1.15.0설치된 python 패키지를 확인해보자.
hadoop@hadoop-001:~$ pip list
Package Version
--------------- -------------------
certifi 2020.12.5
pip 21.0.1
python-dateutil 2.8.1
python-magic 0.4.22
s3cmd 2.1.0
setuptools 52.0.0.post20210125
six 1.15.0
wheel 0.36.22. s3cmd 구성
설치를 완료했으면 Object Storage (업로드 할 곳) 환경을 구성해야한다.
hadoop@hadoop-001:~$ s3cmd --configure접속할 Object Storage의 Access Key, Secret Key, Endpoint 등을 입력한다.
Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.
Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: <접속할 Access Key 입력>
Secret Key: <접속할 Secret Key 입력>
Default Region [US]: KR
Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]: <엔드 포인트 입력 - 예) s3.example.com>
Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: s3.example.com/%(bucket)
Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: airnd
Path to GPG program [/usr/bin/gpg]:
When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]: yes
On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name:
...
Test access with supplied credentials? [Y/n] y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)
Now verifying that encryption works...
Success. Encryption and decryption worked fine :-)
Save settings? [y/N] y
Configuration saved to '/home/s3cmd/.s3cfg'
s3cmd 사용방법
Bucket 리스트 조회
hadoop@hadoop-001:~$ s3cmd ls
2020-01-09 07:39 s3://example-bucket
2020-01-09 07:39 s3://example2-bucket
2020-01-09 07:39 s3://example3-bucketBucket 삭제
hadoop@hadoop-001:~$ s3cmd rb s3://example3-bucketObject 업로드
hadoop@hadoop-001:~$ s3cmd put test.text s3://example-bucket/
upload: './test.text' -> 's3://example-bucket/' [1 of 1]
5 of 5 100% in 0s 64.89 B/s doneObject 업로드 (디렉토리 전체)
recursive 옵션을 활용한다.
hadoop@hadoop-001:~$ s3cmd put ./test_dir s3://example-bucket/test_dir/ --recursive
upload: './test_dir/test.text' -> 's3://example-bucket/test_dir/test.text' [1 of 1]
...Object 리스트
hadoop@hadoop-001:~$ s3cmd ls s3://example-bucket/
2022-04-26 13:00 0 s3://example-bucket/test.txtObject 조회
hadoop@hadoop-001:~$ s3cmd get s3://example-bucket/test.txtObject 삭제
hadoop@hadoop-001:~$ s3cmd rm s3://example-bucket/test.txt728x90
반응형
