S3 Amazon Storage

Amazon Simple Storage Service (Amazon S3) SDK實作

陳志祥 2019/12/23 14:08:03

1 1012

1.前言

Amazon S3，全名為亞馬遜簡易儲存服務（Amazon Simple Storage Service），是亞馬遜公司利用其亞馬遜網路服務系統所提供的網路線上儲存服務。經由Web服務介面，包括REST、SOAP與BitTorrent，用戶能夠輕易把檔案儲存到網路伺服器上。目前雲端儲存已為多數人所選的保存資料方式之一，應用於企業中，較傳統硬碟儲存方式有幾項比較出色的優點，不用維護硬體省去人力成本，不怕硬體損壞導致資料遺失、專案結束時可以隨時關閉服務，在企業專案應用考量下，多了一個儲存資料選項。

S3 雖然是雲端服務，最基本的操作免不了要從瀏覽器登入AWS，點選到S3，手動上傳或是下載檔案，其實AWS S3 有提供各種SDK，可讓用戶透過程式實作，快速的存取S3 檔案，也可便利的應用到各種企業專案中。本文將從建立S3儲存貯體開始，以圖文說明範例操作，介紹如何透過Java SDK進行檔案上傳、讀取與刪除。

2.環境

JDK: 1.8

Build Tool: Maven 3

IDE: IntelliJ IDEA

3.實作

首先先設定個人S3服務，若尚未註冊AWS必須先至AWS進行註冊， https://aws.amazon.com/tw/console/，若已經註冊則直接登入，並在服務中點選S3，開啟並設定該服務

進入S3儲存貯體畫面，依據步驟建立一個新的儲存貯體

不同的區域流量收費標準不同

勾選儲存貯體公開存取設定

儲存貯體建立完成如下

再來我們需要建立使用者，並授與S3讀寫權限，於服務中選擇IAM

選擇將提供給使用者在AWS服務的權限，可以自訂政策，也可以直接套用AWS提供的政策

設定完成後，即可取得該使用者在AWS上，使用權限的金鑰

S3設定完成，再來進行程式撰寫，建立一個Maven Project，並引用IBM SDK

package com.thinkpower.tanktest;

import com.ibm.cloud.objectstorage.ClientConfiguration;
import com.ibm.cloud.objectstorage.auth.AWSCredentials;
import com.ibm.cloud.objectstorage.auth.AWSStaticCredentialsProvider;
import com.ibm.cloud.objectstorage.auth.BasicAWSCredentials;
import com.ibm.cloud.objectstorage.client.builder.AwsClientBuilder;
import com.ibm.cloud.objectstorage.services.s3.AmazonS3;
import com.ibm.cloud.objectstorage.services.s3.AmazonS3ClientBuilder;
import com.ibm.cloud.objectstorage.services.s3.model.*;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;

/**
 * Created by tank on 2019/12/11.
 */
public class S3Service {


    private AmazonS3 _cos;

    public S3Service(String access_key, String secret_key, String endpoint_url, String location) {
        AWSCredentials credentials;
        credentials = new BasicAWSCredentials(access_key, secret_key);
        ClientConfiguration clientConfig = new ClientConfiguration().withRequestTimeout(600000);
        clientConfig.setUseTcpKeepAlive(true);

        _cos = AmazonS3ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(credentials))
                .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(endpoint_url, location)).withPathStyleAccessEnabled(true)
                .withClientConfiguration(clientConfig).build();
    }

    public void uploadToS3(String subFolder, String objectKey, String fileFullPath) {
        System.out.println("put object : ");
        System.out.println("    key : "+objectKey);
        System.out.println("    folder : "+subFolder);
        _cos.putObject(
                subFolder, // the name of the destination bucket
                objectKey, // the object key/Users/tank/Downloads
                new File(fileFullPath) // the file name and path of the object to be uploaded
        );
        System.out.println("--------------------------------------------");
    }

    public void getObjectInfo(String subFolder, String objectKey) {
        System.out.println("show Object Info :");
        S3Object s3Object = _cos.getObject(subFolder ,objectKey);
        System.out.println("    "+s3Object.toString());
        System.out.println("--------------------------------------------");
    }

    public boolean isExistObject(String subFolder, String objectKey) {
        System.out.println("Object is exists : ");
        System.out.println("    folder : "+subFolder);
        System.out.println("    object key : "+objectKey);
        boolean result = _cos.doesObjectExist(subFolder , objectKey);
        System.out.println("    Exists : "+result);
        System.out.println("--------------------------------------------");
        return result;
    }

    public void deleteByKey(String subFolder, String objectKey) {
        System.out.println("delete object :");
        System.out.println("    folder :"+subFolder);
        System.out.println("    object key :"+objectKey);
        _cos.deleteObject(subFolder, objectKey);
        System.out.println("--------------------------------------------");
    }

    public void deleteUploadedFile(String fileAbsName) {
        File file = new File(fileAbsName);
        file.delete();
    }

    public List<String> listWarFileByPath(String path) throws IOException {
        Stream<Path> walk = Files.walk(Paths.get(path));
        List<String> result = walk.map(x -> x.getFileName().toString())
                .filter(f -> f.endsWith(".png"))
                .collect(Collectors.toList());

        result.forEach(System.out::println);
        return result;
    }

    public void uploadWarFileOnDesktop(String folderName, String filePath) throws IOException {
        List<String> fileList = listWarFileByPath(filePath);
        for (String fileKey : fileList) {
            String absFileName = filePath + fileKey;
            uploadToS3(folderName, fileKey, absFileName);
            deleteUploadedFile(absFileName);
        }
    }

    public void downloadFileFromS3(String folderName, String putKey, String filename) {
        System.out.println("downlaod start ...");
        GetObjectRequest request = new // create a new request to get an object
                GetObjectRequest( // request the new object by identifying
                folderName, // the name of the bucket
                putKey // the name of the object
        );

        _cos.getObject( // write the contents of the object
                request, // using the request that was just created
                new File(filename) // to write to a new file
        );
    }
}

程式測試，將圖片上傳至S3，之後進行查詢，然後再刪除，結果輸出如下：