Spring Boot S3 Integration – File Upload / Download

vIns

6 years ago

Overview:

In this tutorial, I would like to demo Spring Boot S3 Integration & how we could upload/download files to/from a AWS S3 bucket easily!

Spring Boot S3 Integration:

Most of us are using AWS cloud for the Spring Boot applications. Often times we will also have requirements to access files to/from a S3 bucket. However adding AWS S3 specific code in your service classes would make the local development / testing difficult. The code would be tightly coupled with AWS S3. So any developer who likes to run the service in their local needs to setup AWS credentials etc which could be annoying.

The idea is to avoid AWS specific dependency in the code – so that our application will work everywhere – local or cloud!

The goal of this tutorial is to

Develop and run the application in the local using local file system without using S3
Run the same application in AWS by using S3 as file system

Project Setup:

Create a simple Spring Boot project as shown here with the below dependencies.

I have my project structure as shown below.

Controller:

My controller has 2 endpoints. 1 is creating the file and another endpoint for accessing the file.

import com.vinsguru.service.FileService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import java.io.IOException;

@RestController
public class S3Resource {

    @Autowired
    private FileService fileService;

    @GetMapping("/demo/get/{id}")
    public ResponseEntity<ByteArrayResource> getFile(@PathVariable String id) throws IOException {
        final ByteArrayResource byteArrayResource = new ByteArrayResource(this.fileService.getFile(id));
        return ResponseEntity
                .ok()
                .contentLength(byteArrayResource.contentLength())
                .contentType(MediaType.parseMediaType("application/octet-stream"))
                .header("Content-Disposition", "attachment; filename=" + id + ".csv")
                .body(byteArrayResource);
    }

    @GetMapping("/demo/put/{id}")
    public void putFile(@PathVariable String id) throws IOException {
        this.fileService.putFile(id);
    }
    
}

Accessing File System – Service:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.core.io.ResourceLoader;
import org.springframework.core.io.WritableResource;
import org.springframework.stereotype.Service;
import org.springframework.util.FileCopyUtils;

import java.io.IOException;
import java.io.OutputStream;

@Service
public class FileService {

    private static final String EXTENSION = ".csv";

    @Autowired
    private ResourceLoader resourceLoader;

    @Value("${file.location}")
    private String location;

    public byte[] getFile(final String id) throws IOException {
        System.out.println("File to be accessed :: " + getLocation(id));
        Resource resource = this.resourceLoader.getResource(getLocation(id));
        return FileCopyUtils.copyToByteArray(resource.getInputStream());
    }

    public void putFile(final String id) throws IOException {
        System.out.println("File to be placed :: " + getLocation(id));
        OutputStream outputStream = ((WritableResource) this.resourceLoader.getResource(getLocation(id))).getOutputStream();
        FileCopyUtils.copy(createFile(id), outputStream);
    }

    private byte[] createFile(final String id){
        String txt = "This file is created for the user," + id;
        return txt.getBytes();
    }

    private String getLocation(final String id){
        //location/123.csv
        return this.location + "/" + id + EXTENSION;
    }
 }

Profiles:

Now I create 2 different profiles in my Spring Boot application.

application-local.properties
application-aws.properties

Local Profile – Local File System:

My application-local.properties has below content.

file.location=file:/home/vins/Documents/s3demo

Now I can run the application in the local using local profile. Accessing below endpoints creates and gives us the file from the above mentioned local file system path.

http://localhost:8080/demo/put/123
http://localhost:8080/demo/get/123

AWS Profile – Spring Boot S3:

My application-aws.properties has below content. vinsguru-demo is the S3 bucket name.

file.location=s3://vinsguru-demo/s3-demo
cloud.aws.region.static=us-east-1
cloud.aws.stack.auto=false

I can run the application in the AWS cloud using the aws profile without doing any other code change. Accessing below endpoints creates file in S3 and gives us the file from S3. Here we assume that the bucket is present and app has permission to put and get objects.

http://[domain]/demo/put/123
http://[domain]/demo/get/123

Summary:

We were able to successfully demonstrate Spring Boot S3 integration just by using maven dependency – but not touching the service layer. It makes the code look clean & It also makes the local development process easier.