How to Manage GCS with @aws-sdk/client-s3

Table of Con­tents

• Pre­req­ui­sites
• Step-by-Step Set­up
• Ver­i­fi­ca­tion & Val­i­da­tion
• Trou­bleshoot­ing & Pit­falls
• FAQ
• Se­cu­ri­ty Best Prac­tices
• Use­ful Links

Pre­req­ui­sites

• A Google Cloud ac­count with billing en­abled
• An ex­ist­ing GCS buck­et (ex­am­ple: gcs-snapix-test)
• Ac­cess to Google Cloud Con­sole
• [Abil­i­ty to use Cloud Shell (built-in ter­mi­nal)]
• [Node.js and the @aws-sdk/client-s3 pack­age in­stalled lo­cal­ly or on your serv­er]

Note: Re­place all cre­den­tials with place­hold­ers like YOUR_SECRET_VALUE_HERE.

Step-by-Step Set­up

1. Open Cloud Shell in the Google Cloud Con­sole

1. Nav­i­gate to Google Cloud Con­sole.
2. Click the ter­mi­nal icon in the top-right cor­ner to ac­ti­vate Cloud Shell.
3. Run:

   gsutil ls
   

This ver­i­fies your buck­et ex­ists.

2. Make the GCS Buck­et Pub­lic

En­able uni­form buck­et-lev­el ac­cess and al­low the pub­lic to view ob­jects:

BUCKET_NAME=gcs-snapix-test

# Enable uniform access
gsutil uniformbucketlevelaccess set on gs://$BUCKET_NAME

# Grant read-only access to all users
gsutil iam ch allUsers:objectViewer gs://$BUCKET_NAME

Now all ob­jects in the buck­et can be ac­cessed via URLs like:

https://storage.googleapis.com/gcs-snapix-test/test-117.webp

⚠️ Warn­ing: This makes every ob­ject in this buck­et pub­licly ac­ces­si­ble. Don’t store sen­si­tive data here.

3. Cre­ate a Buck­et-Scoped Ser­vice Ac­count

1. Go to IAM & Ad­min → IAM in the Google Cloud Con­sole.

2. Re­move any broad roles like Ed­i­tor from this ser­vice ac­count.

3. Go to Cloud Stor­age → Buck­ets → gcs-snapix-test → Per­mis­sions.

4. Click + Grant Ac­cess:

   • En­ter your ser­vice ac­count email.
   • Choose Stor­age Ob­ject Ad­min (or Stor­age Ob­ject Cre­ator for up­load-only ac­cess).
   • Save changes.

Con­firm per­mis­sions with:

gsutil iam get gs://gcs-snapix-test

4. Cre­ate HMAC Cre­den­tials

Gen­er­ate HMAC keys for this ser­vice ac­count:

SERVICE_ACCOUNT=[email protected]
gcloud storage hmac create $SERVICE_ACCOUNT

Ex­am­ple out­put:

metadata:
  accessId: GOOG1E257VODVI4M2JWSOX5WX3QKMZBKE5YRM74Z4SUEWADE5QSBWCB5SIWT7
secret: YOUR_SECRET_VALUE_HERE

• accessId → accessKeyId in AWS SDK.
• secret → secretAccessKey in AWS SDK.

⚠️ Im­por­tant: The se­cret is only shown once. Save it se­cure­ly.

5. Con­fig­ure @aws-sdk/client-s3 for GCS

In­stall the AWS SDK for JavaScript (v3):

npm install @aws-sdk/client-s3

Then con­fig­ure it to use GCS:

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
import fs from "fs";

const s3 = new S3Client({
  region: "auto",
  endpoint: "https://storage.googleapis.com",
  credentials: {
    accessKeyId: "YOUR_ACCESS_KEY_VALUE_HERE", // from accessId
    secretAccessKey: "YOUR_SECRET_VALUE_HERE", // from secret
  },
  forcePathStyle: true,
});

6. Up­load a File via SDK

async function uploadFile() {
  const fileContent = fs.readFileSync("test-117.webp");

  const uploadParams = {
    Bucket: "gcs-snapix-test",
    Key: "test-117.webp",
    Body: fileContent,
  };

  await s3.send(new PutObjectCommand(uploadParams));
  console.log("Uploaded successfully!");
}

uploadFile();

Ver­i­fi­ca­tion & Val­i­da­tion

• Run:

  gsutil ls gs://gcs-snapix-test
  

  Your file should be list­ed.

• Open:

  https://storage.googleapis.com/gcs-snapix-test/test-117.webp
  

  The file should load in your brows­er.

Trou­bleshoot­ing & Pit­falls

FAQ

Q: Can I use the AWS SDK for all GCS fea­tures? A: No. GCS’s S3 in­ter­op­er­abil­i­ty sup­ports ba­sic op­er­a­tions like up­load­ing, down­load­ing, and list­ing ob­jects. Ad­vanced fea­tures may re­quire Google’s na­tive SDK.

Q: Why do I need HMAC keys? A: The AWS SDK uses sig­na­ture-based au­then­ti­ca­tion that re­quires ac­cess keys and se­crets, pro­vid­ed by HMAC cre­den­tials in GCS.

Se­cu­ri­ty Best Prac­tices

1. Use least priv­i­lege

   • Re­strict the ser­vice ac­count to only the buck­ets it needs.
   • Use roles like Storage Object Creator or Storage Object Viewer for tighter con­trol.

2. Ro­tate HMAC keys reg­u­lar­ly

   • Cre­ate a new key, up­date apps, and then delete the old one.

3. Nev­er hard­code se­crets

   • Store cre­den­tials in en­vi­ron­ment vari­ables or a se­crets man­ag­er (e.g., GCP Se­cret Man­ag­er, AWS Se­crets Man­ag­er).

4. Mon­i­tor ac­cess logs

   • En­able Cloud Au­dit Logs to see who ac­cessed what.

5. Avoid pub­lic buck­ets for sen­si­tive data

   • Use signed URLs or iden­ti­ty-aware prox­ies for con­trolled ac­cess.

6. Use sep­a­rate ser­vice ac­counts per app

   • Makes it eas­i­er to au­dit and re­voke ac­cess if need­ed.

Use­ful Links

• Google Cloud Stor­age In­ter­op­er­abil­i­ty
• AWS SDK for JavaScript v3 – S3 Client
• Cloud Shell Doc­u­men­ta­tion