Spring Boot + AWS S3 Upload File
In this tutorial, we will develop AWS Simple Storage Service (S3) together with Spring Boot Rest API service to upload file on S3.
Amazon S3 Tutorial :
What is S3?
Amazon Simple Storage Service (Amazon S3) is an object storage service that provides industry-leading scalability, data availability, security, and performance.
The service can be used as online backup and archiving of data and applications on Amazon Web Services (AWS).
AWS Core S3 Concepts
In 2006, S3 was one of the first services provided by AWS. Many features have been introduced since then, but the core principles of S3 remain Buckets and Objects.
AWS Buckets
Buckets are containers for objects that we choose to store. It is necessary to remember that S3 allows the bucket name to be globally unique.AWS Objects
Objects are the actual items that we store in S3. They are marked by a key, which is a sequence of Unicode characters with a maximum length of 1,024 bytes in UTF-8 encoding.
Prerequisites
First Create Bucket on Amazon S3 and then Generate Credentials(accessKey and secretKey) to access AWS S3 bucket.
Take a look at our suggested posts:
Let's start developing AWS S3 + Spring Boot application.
Create Spring Boot Project
Create project from Spring Initializr as shown below and extract the project into Eclipse/IntelliJ IDE.
Project Structure
Maven Dependency
Add spring-cloud-starter-aws, which have spring-cloud-aws-context and spring-cloud-aws-autoconfigure dependencies.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.10.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.techgeeknext</groupId>
<artifactId>springboot-aws-s3</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>springboot-aws-s3</name>
<description>Demo project for Spring Boot AWS S3</description>
<properties>
<java.version>1.8</java.version>
<spring-cloud.version>Hoxton.SR11</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-aws</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
Spring Cloud AWS Configuration
Get "Access Key ID" and "Secret Access Key" from AWS S3 Security Credential and add in application.yml.
#Application properties
server:
port: 8081
#AWS properties
cloud:
aws:
region:
static: ap-south-1
stack:
auto: false
credentials:
accessKey: AKIA32CIHYHAOT2KHV
secretKey: fcmvvq1hcDoSWmHSem6VRc6GdHw2081lUawajH
application:
bucket:
name: techgeeknextbucket
logging:
level:
com:
amazonaws:
util:
EC2MetadataUtils: error
Add this logging.level.com.amazonaws.util.EC2MetadataUtils entry in application.yml to get rid of below exception.
fails to connect to service endpoint locally
WARN 22462 --- [ restartedMain] com.amazonaws.util.EC2MetadataUtils: Unable to retrieve the requested metadata (/latest/meta-data/instance-id). Failed to connect to service endpoint:
com.amazonaws.SdkClientException: Failed to connect to service endpoint:
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100) ~[aws-java-sdk-core-1.11.699.jar:na]
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:70) ~[aws-java-sdk-core-1.11.699.jar:na]
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:75) ~[aws-java-sdk-core-1.11.699.jar:na]
at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:62) ~[aws-java-sdk-core-1.11.699.jar:na]
at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:400) ~[aws-java-sdk-core-1.11.699.jar:na]
at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:369) ~[aws-java-sdk-core-1.11.699.jar:na]
at org.springframework.cloud.aws.context.support.env.AwsCloudEnvironmentCheckUtils.isRunningOnCloudEnvironment(AwsCloudEnvironmentCheckUtils.java:38)
region.static in the properties file else you will get the exception related to region as given below.
Caused by: java.lang.IllegalArgumentException: The region 'us-east-1' is not a valid region!
at org.springframework.cloud.aws.core.region.StaticRegionProvider.<init>(StaticRegionProvider.java:47) Amazon S3 Client Configuration
The main class for communicating with S3 is com.amazonaws.services.s3.AmazonS3.
To access the Amazon S3 web service, we must first establish a client connection.
We will use BasicAWSCredentials AmazonS3 interface with AWS Access Key and AWS Secret Key and then configure
the AmazonS3 client by passing this credential to AWSStaticCredentialsProvider as shown below.
package com.techgeeknext.springbootawss3.config;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class AwsS3ClientConfig {
@Value("${cloud.aws.credentials.accessKey}")
private String awsId;
@Value("${cloud.aws.credentials.secretKey}")
private String awsKey;
@Value("${cloud.aws.region.static}")
private String region;
@Bean
public AmazonS3 s3client() {
BasicAWSCredentials awsCredentials = new BasicAWSCredentials(awsId, awsKey);
AmazonS3 amazonS3Client = AmazonS3ClientBuilder.standard()
.withRegion(Regions.fromName(region))
.withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
.build();
return amazonS3Client;
}
}
Implement AWS S3 Services
Create the S3BucketStorageService to implement the function to upload file in AWS S3 Bucket.
package com.techgeeknext.springbootawss3.service;
import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.model.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import java.io.IOException;
@Service
public class S3BucketStorageService {
private Logger logger = LoggerFactory.getLogger(S3BucketStorageService.class);
@Autowired
private AmazonS3 amazonS3Client;
@Value("${application.bucket.name}")
private String bucketName;
/**
* Upload file into AWS S3
*
* @param keyName
* @param file
* @return String
*/
public String uploadFile(String keyName, MultipartFile file) {
try {
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(file.getSize());
amazonS3Client.putObject(bucketName, keyName, file.getInputStream(), metadata);
return "File uploaded: " + keyName;
} catch (IOException ioe) {
logger.error("IOException: " + ioe.getMessage());
} catch (AmazonServiceException serviceException) {
logger.info("AmazonServiceException: "+ serviceException.getMessage());
throw serviceException;
} catch (AmazonClientException clientException) {
logger.info("AmazonClientException Message: " + clientException.getMessage());
throw clientException;
}
return "File not uploaded: " + keyName;
}
}
RestAPI to Upload File on AWS S3
Create the RestController class to test the upload file operation in AWS S3 bucket.
package com.techgeeknext.springbootawss3.controller;
import com.techgeeknext.springbootawss3.service.S3BucketStorageService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import java.util.List;
@RestController
public class S3BucketStorageController {
@Autowired
S3BucketStorageService service;
@GetMapping("/list/files")
public ResponseEntity<List<String>> getListOfFiles() {
return new ResponseEntity<>(service.listFiles(), HttpStatus.OK);
}
@PostMapping("/file/upload")
public ResponseEntity<String> uploadFile(@RequestParam("fileName") String fileName,
@RequestParam("file") MultipartFile file) {
return new ResponseEntity<>(service.uploadFile(fileName, file), HttpStatus.OK);
}
}
Test AWS S3 Upload Operation
Now, run the Spring Boot application.
Upload File on AWS S3 Bucket
Use POST method with url http://localhost:8081/file/upload, select file and provide filename.
- Verify on AWS S3 Bucket.
Download Source Code
The full source code for this article can be found on below.Download it here - Spring Cloud: AWS S3 Example