---
title: "Replit Object Storage with AWS S3: Python and TypeScript Setup"
description: "Use AWS S3 as object storage for a Replit app: create the bucket, an IAM user with tight permissions, and Replit Secrets, then upload and download files with boto3 or the AWS SDK v3."
image: "https://www.sethserver.com/static/images/ai-posts/connecting-your-replit-app-to-aws-s3-social.jpg"
---

# Replit Object Storage with AWS S3: Python and TypeScript Setup

S3 is one of those AWS services that just works. It's been around since 2006, it's battle-tested, and it scales infinitely without you having to think about disk provisioning or RAID configurations. Need to store files? S3. Need to host static assets? S3. Need cheap storage for data you rarely access? Also S3.

I use S3 constantly for storing JSON and XML documents when training ML algorithms. When you're doing infrequent reads and writes — pull a dataset, train a model, push results back — S3 is perfect. It's dirt cheap for storage, you only pay for what you transfer, and you don't need to keep a database running 24/7 for data that gets touched once a week. For batch ML workflows, it's the obvious choice.

Let's get your Replit app connected.

## The Short Version

You need an S3 bucket, IAM credentials with the right permissions, and a few lines of code. Unlike RDS, there's no networking to configure — S3 is a public service (your data isn't public, but the API endpoint is). No security groups, no VPCs, no public accessibility toggles.

## Step 1: Create an S3 Bucket

Head to the S3 console in AWS and click "Create bucket."

- **Bucket name:** Must be globally unique across all of AWS. Use something like `yourcompany-replit-data` or `yourproject-ml-datasets`. You can't change this later.
- **Region:** Pick one close to where your Replit app runs, or where your users are. `us-east-1` is the default and usually fine.
- **Object Ownership:** Keep "ACLs disabled" — you'll manage access through IAM policies, which is the right way to do it.
- **Block Public Access:** Leave everything checked. You do not want your bucket accidentally public. Trust me on this one.
- **Versioning:** Enable it if you want to keep old versions of files. Useful for ML datasets where you might need to reproduce results with the exact data you trained on.
- **Encryption:** Default encryption with SSE-S3 is fine for most cases.

Click "Create bucket" and you're done with this part.

## Step 2: Create IAM Credentials

Your Replit app needs credentials to access S3. We'll create an IAM user with minimal permissions.

1. Go to IAM in the AWS Console
2. Click "Users" then "Create user"
3. Name it something like `replit-s3-access`
4. Click "Next"
5. Choose "Attach policies directly"
6. For now, search for `AmazonS3FullAccess` — we'll tighten this up in a moment
7. Create the user
8. Click on the user, go to "Security credentials"
9. Click "Create access key"
10. Choose "Application running outside AWS"
11. Save the **Access Key ID** and **Secret Access Key**

### Tighten Those Permissions

`AmazonS3FullAccess` gives access to every bucket in your account. That's too broad. Create a custom policy that limits access to just your bucket:

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::your-bucket-name",
        "arn:aws:s3:::your-bucket-name/*"
      ]
    }
  ]
}
```

Replace `your-bucket-name` with your actual bucket name. The first ARN allows listing the bucket contents, the second allows operations on objects inside it.

Go back to your IAM user, detach the `AmazonS3FullAccess` policy, and attach your new custom policy instead.

## Step 3: Set Up Secrets in Replit

Add your AWS credentials to Replit's Secrets:

```
AWS_ACCESS_KEY_ID = your_access_key_id
AWS_SECRET_ACCESS_KEY = your_secret_access_key
AWS_REGION = us-east-1
S3_BUCKET_NAME = your-bucket-name
```

## Step 4: Connect From Your Code

### Python with boto3

**Basic operations:**

```
import os
import json
import boto3
from botocore.exceptions import ClientError

s3 = boto3.client(
    's3',
    region_name=os.environ.get('AWS_REGION', 'us-east-1'),
    aws_access_key_id=os.environ.get('AWS_ACCESS_KEY_ID'),
    aws_secret_access_key=os.environ.get('AWS_SECRET_ACCESS_KEY')
)

BUCKET = os.environ.get('S3_BUCKET_NAME')

def upload_file(local_path: str, s3_key: str) -> bool:
    """Upload a file to S3."""
    try:
        s3.upload_file(local_path, BUCKET, s3_key)
        print(f"Uploaded {local_path} to s3://{BUCKET}/{s3_key}")
        return True
    except ClientError as e:
        print(f"Upload failed: {e}")
        return False

def download_file(s3_key: str, local_path: str) -> bool:
    """Download a file from S3."""
    try:
        s3.download_file(BUCKET, s3_key, local_path)
        print(f"Downloaded s3://{BUCKET}/{s3_key} to {local_path}")
        return True
    except ClientError as e:
        print(f"Download failed: {e}")
        return False

def list_objects(prefix: str = '') -> list:
    """List objects in the bucket with optional prefix filter."""
    try:
        response = s3.list_objects_v2(Bucket=BUCKET, Prefix=prefix)
        return [obj['Key'] for obj in response.get('Contents', [])]
    except ClientError as e:
        print(f"List failed: {e}")
        return []

def delete_object(s3_key: str) -> bool:
    """Delete an object from S3."""
    try:
        s3.delete_object(Bucket=BUCKET, Key=s3_key)
        print(f"Deleted s3://{BUCKET}/{s3_key}")
        return True
    except ClientError as e:
        print(f"Delete failed: {e}")
        return False
```

**Working with JSON documents (my bread and butter for ML work):**

```
import os
import json
import boto3
from typing import Any
from datetime import datetime

s3 = boto3.client(
    's3',
    region_name=os.environ.get('AWS_REGION', 'us-east-1'),
    aws_access_key_id=os.environ.get('AWS_ACCESS_KEY_ID'),
    aws_secret_access_key=os.environ.get('AWS_SECRET_ACCESS_KEY')
)

BUCKET = os.environ.get('S3_BUCKET_NAME')

def save_json(data: Any, s3_key: str) -> str:
    """Save a Python object as JSON to S3."""
    json_bytes = json.dumps(data, indent=2, default=str).encode('utf-8')
    s3.put_object(Bucket=BUCKET, Key=s3_key, Body=json_bytes, ContentType='application/json')
    return f"s3://{BUCKET}/{s3_key}"

def load_json(s3_key: str) -> Any:
    """Load a JSON document from S3."""
    response = s3.get_object(Bucket=BUCKET, Key=s3_key)
    return json.loads(response['Body'].read().decode('utf-8'))

def save_training_dataset(dataset: list, model_name: str) -> str:
    """Save a training dataset with timestamp for reproducibility."""
    timestamp = datetime.utcnow().strftime('%Y%m%d_%H%M%S')
    s3_key = f"datasets/{model_name}/{timestamp}/training_data.json"
    return save_json(dataset, s3_key)

def save_model_metrics(metrics: dict, model_name: str, version: str) -> str:
    """Save model training metrics."""
    s3_key = f"models/{model_name}/{version}/metrics.json"
    return save_json(metrics, s3_key)

# Example ML workflow
training_data = [
    {"input": [1, 2, 3], "label": "A"},
    {"input": [4, 5, 6], "label": "B"},
    # ... thousands more rows
]

# Save dataset before training
dataset_path = save_training_dataset(training_data, "classifier_v1")
print(f"Dataset saved to {dataset_path}")

# After training, save metrics
metrics = {
    "accuracy": 0.94,
    "f1_score": 0.92,
    "training_time_seconds": 3842,
    "hyperparameters": {"learning_rate": 0.001, "epochs": 100}
}
metrics_path = save_model_metrics(metrics, "classifier_v1", "20240115")
print(f"Metrics saved to {metrics_path}")
```

**Working with XML documents:**

```
import os
import boto3
import xml.etree.ElementTree as ET
from typing import Optional

s3 = boto3.client(
    's3',
    region_name=os.environ.get('AWS_REGION', 'us-east-1'),
    aws_access_key_id=os.environ.get('AWS_ACCESS_KEY_ID'),
    aws_secret_access_key=os.environ.get('AWS_SECRET_ACCESS_KEY')
)

BUCKET = os.environ.get('S3_BUCKET_NAME')

def save_xml(root: ET.Element, s3_key: str) -> str:
    """Save an XML ElementTree to S3."""
    xml_bytes = ET.tostring(root, encoding='utf-8', xml_declaration=True)
    s3.put_object(Bucket=BUCKET, Key=s3_key, Body=xml_bytes, ContentType='application/xml')
    return f"s3://{BUCKET}/{s3_key}"

def load_xml(s3_key: str) -> ET.Element:
    """Load an XML document from S3."""
    response = s3.get_object(Bucket=BUCKET, Key=s3_key)
    return ET.fromstring(response['Body'].read())

# Example: save training configuration as XML
config = ET.Element("training_config")
ET.SubElement(config, "model_type").text = "random_forest"
ET.SubElement(config, "n_estimators").text = "100"
ET.SubElement(config, "max_depth").text = "10"

save_xml(config, "configs/rf_config.xml")
```

### TypeScript with AWS SDK v3

**Basic operations:**

```
import {
  S3Client,
  PutObjectCommand,
  GetObjectCommand,
  DeleteObjectCommand,
  ListObjectsV2Command
} from '@aws-sdk/client-s3';
import { Readable } from 'stream';
import * as fs from 'fs';

const s3Client = new S3Client({
  region: process.env.AWS_REGION || 'us-east-1',
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!
  }
});

const BUCKET = process.env.S3_BUCKET_NAME!;

async function streamToString(stream: Readable): Promise<string> {
  const chunks: Buffer[] = [];
  for await (const chunk of stream) {
    chunks.push(Buffer.from(chunk));
  }
  return Buffer.concat(chunks).toString('utf-8');
}

async function uploadFile(localPath: string, s3Key: string): Promise<void> {
  const fileContent = fs.readFileSync(localPath);
  const command = new PutObjectCommand({
    Bucket: BUCKET,
    Key: s3Key,
    Body: fileContent
  });
  await s3Client.send(command);
  console.log(`Uploaded ${localPath} to s3://${BUCKET}/${s3Key}`);
}

async function downloadFile(s3Key: string, localPath: string): Promise<void> {
  const command = new GetObjectCommand({
    Bucket: BUCKET,
    Key: s3Key
  });
  const response = await s3Client.send(command);
  const content = await streamToString(response.Body as Readable);
  fs.writeFileSync(localPath, content);
  console.log(`Downloaded s3://${BUCKET}/${s3Key} to ${localPath}`);
}

async function listObjects(prefix: string = ''): Promise<string[]> {
  const command = new ListObjectsV2Command({
    Bucket: BUCKET,
    Prefix: prefix
  });
  const response = await s3Client.send(command);
  return (response.Contents || []).map(obj => obj.Key!);
}

async function deleteObject(s3Key: string): Promise<void> {
  const command = new DeleteObjectCommand({
    Bucket: BUCKET,
    Key: s3Key
  });
  await s3Client.send(command);
  console.log(`Deleted s3://${BUCKET}/${s3Key}`);
}
```

**Working with JSON documents:**

```
import {
  S3Client,
  PutObjectCommand,
  GetObjectCommand
} from '@aws-sdk/client-s3';
import { Readable } from 'stream';

const s3Client = new S3Client({
  region: process.env.AWS_REGION || 'us-east-1',
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!
  }
});

const BUCKET = process.env.S3_BUCKET_NAME!;

async function streamToString(stream: Readable): Promise<string> {
  const chunks: Buffer[] = [];
  for await (const chunk of stream) {
    chunks.push(Buffer.from(chunk));
  }
  return Buffer.concat(chunks).toString('utf-8');
}

async function saveJson<T>(data: T, s3Key: string): Promise<string> {
  const command = new PutObjectCommand({
    Bucket: BUCKET,
    Key: s3Key,
    Body: JSON.stringify(data, null, 2),
    ContentType: 'application/json'
  });
  await s3Client.send(command);
  return `s3://${BUCKET}/${s3Key}`;
}

async function loadJson<T>(s3Key: string): Promise<T> {
  const command = new GetObjectCommand({
    Bucket: BUCKET,
    Key: s3Key
  });
  const response = await s3Client.send(command);
  const content = await streamToString(response.Body as Readable);
  return JSON.parse(content) as T;
}

// Example interfaces for ML workflow
interface TrainingRecord {
  input: number[];
  label: string;
}

interface ModelMetrics {
  accuracy: number;
  f1Score: number;
  trainingTimeSeconds: number;
  hyperparameters: Record<string, number | string>;
}

async function saveTrainingDataset(
  dataset: TrainingRecord[],
  modelName: string
): Promise<string> {
  const timestamp = new Date().toISOString().replace(/[-:]/g, '').split('.')[0];
  const s3Key = `datasets/${modelName}/${timestamp}/training_data.json`;
  return saveJson(dataset, s3Key);
}

async function saveModelMetrics(
  metrics: ModelMetrics,
  modelName: string,
  version: string
): Promise<string> {
  const s3Key = `models/${modelName}/${version}/metrics.json`;
  return saveJson(metrics, s3Key);
}

// Usage
const trainingData: TrainingRecord[] = [
  { input: [1, 2, 3], label: 'A' },
  { input: [4, 5, 6], label: 'B' },
];

const datasetPath = await saveTrainingDataset(trainingData, 'classifier_v1');
console.log(`Dataset saved to ${datasetPath}`);

const metrics: ModelMetrics = {
  accuracy: 0.94,
  f1Score: 0.92,
  trainingTimeSeconds: 3842,
  hyperparameters: { learningRate: 0.001, epochs: 100 }
};

const metricsPath = await saveModelMetrics(metrics, 'classifier_v1', '20240115');
console.log(`Metrics saved to ${metricsPath}`);
```

### A Reusable S3 Storage Class (TypeScript)

```
import {
  S3Client,
  PutObjectCommand,
  GetObjectCommand,
  DeleteObjectCommand,
  ListObjectsV2Command,
  HeadObjectCommand
} from '@aws-sdk/client-s3';
import { Readable } from 'stream';

interface S3StorageConfig {
  bucket: string;
  prefix?: string;
}

class S3Storage {
  private client: S3Client;
  private bucket: string;
  private prefix: string;

  constructor(config: S3StorageConfig) {
    this.client = new S3Client({
      region: process.env.AWS_REGION || 'us-east-1',
      credentials: {
        accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
        secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!
      }
    });
    this.bucket = config.bucket;
    this.prefix = config.prefix || '';
  }

  private fullKey(key: string): string {
    return this.prefix ? `${this.prefix}/${key}` : key;
  }

  private async streamToBuffer(stream: Readable): Promise<Buffer> {
    const chunks: Buffer[] = [];
    for await (const chunk of stream) {
      chunks.push(Buffer.from(chunk));
    }
    return Buffer.concat(chunks);
  }

  async put(key: string, data: string | Buffer, contentType?: string): Promise<void> {
    const command = new PutObjectCommand({
      Bucket: this.bucket,
      Key: this.fullKey(key),
      Body: data,
      ContentType: contentType
    });
    await this.client.send(command);
  }

  async putJson<T>(key: string, data: T): Promise<void> {
    await this.put(key, JSON.stringify(data, null, 2), 'application/json');
  }

  async get(key: string): Promise<Buffer> {
    const command = new GetObjectCommand({
      Bucket: this.bucket,
      Key: this.fullKey(key)
    });
    const response = await this.client.send(command);
    return this.streamToBuffer(response.Body as Readable);
  }

  async getJson<T>(key: string): Promise<T> {
    const buffer = await this.get(key);
    return JSON.parse(buffer.toString('utf-8')) as T;
  }

  async exists(key: string): Promise<boolean> {
    try {
      const command = new HeadObjectCommand({
        Bucket: this.bucket,
        Key: this.fullKey(key)
      });
      await this.client.send(command);
      return true;
    } catch {
      return false;
    }
  }

  async delete(key: string): Promise<void> {
    const command = new DeleteObjectCommand({
      Bucket: this.bucket,
      Key: this.fullKey(key)
    });
    await this.client.send(command);
  }

  async list(subPrefix: string = ''): Promise<string[]> {
    const fullPrefix = this.fullKey(subPrefix);
    const command = new ListObjectsV2Command({
      Bucket: this.bucket,
      Prefix: fullPrefix
    });
    const response = await this.client.send(command);
    return (response.Contents || []).map(obj => obj.Key!);
  }
}

// Usage
const storage = new S3Storage({
  bucket: process.env.S3_BUCKET_NAME!,
  prefix: 'ml-experiments'
});

await storage.putJson('experiment-001/config.json', {
  learningRate: 0.001,
  epochs: 100
});

const config = await storage.getJson<{ learningRate: number; epochs: number }>(
  'experiment-001/config.json'
);
```

## Storage Classes: Saving Money on ML Data

S3 has different storage classes with different pricing. For ML datasets that you access infrequently, you can save serious money:

- **S3 Standard:** Default, good for frequently accessed data
- **S3 Intelligent-Tiering:** Automatically moves data between tiers based on access patterns — set it and forget it
- **S3 Standard-IA (Infrequent Access):** Cheaper storage, small retrieval fee — perfect for training datasets you touch weekly or monthly
- **S3 Glacier Instant Retrieval:** Even cheaper, for data accessed quarterly
- **S3 Glacier Deep Archive:** Cheapest, for archival data you might never need again

For ML work, I typically use **Standard-IA** for datasets and model artifacts. You pay less for storage and the retrieval cost is negligible when you're only pulling data for training runs.

Set the storage class when uploading:

```
s3.put_object(
    Bucket=BUCKET,
    Key=s3_key,
    Body=data,
    StorageClass='STANDARD_IA'
)
```

```
const command = new PutObjectCommand({
  Bucket: BUCKET,
  Key: s3Key,
  Body: data,
  StorageClass: 'STANDARD_IA'
});
```

Or set up lifecycle rules in the S3 console to automatically transition objects to cheaper storage after a certain number of days.

## When Things Go Wrong

**"Access Denied":** Your IAM policy doesn't grant the right permissions, or the bucket name in your policy doesn't match the actual bucket. Double-check both.

**"NoSuchBucket":** Bucket name is wrong or the bucket doesn't exist. Remember bucket names are globally unique — maybe someone else has that name.

**"NoSuchKey":** The object doesn't exist. Check your key path, including any prefixes.

**Slow uploads/downloads:** S3 is fast, but if you're transferring large files, consider using multipart uploads for files over 100MB. boto3's `upload_file` handles this automatically.

**"InvalidAccessKeyId":** Your access key is wrong or has been deactivated. Generate new credentials in IAM.

**"SignatureDoesNotMatch":** Usually means your secret key is wrong, or there's a clock skew issue (rare in cloud environments).

## Organizing Your Bucket

For ML work, I recommend a structure like this:

```
your-bucket/
├── datasets/
│   ├── project-a/
│   │   ├── 20240115_raw/
│   │   ├── 20240115_processed/
│   │   └── 20240120_augmented/
│   └── project-b/
├── models/
│   ├── project-a/
│   │   ├── v1/
│   │   │   ├── weights.h5
│   │   │   ├── config.json
│   │   │   └── metrics.json
│   │   └── v2/
│   └── project-b/
├── experiments/
│   └── hyperparameter-search-001/
└── configs/
    └── training-defaults.json
```

Timestamps in folder names. Version numbers. Clear separation between raw data, processed data, and outputs. Your future self will thank you when you need to reproduce results from six months ago.

## One More Thing

S3 pricing is usage-based — you pay for storage, requests, and data transfer. For small projects, it's basically free. For large ML datasets, it can add up, but it's still way cheaper than keeping an RDS instance running.

The real cost trap is data transfer out of AWS. Downloading data from S3 to your local machine or to Replit (which is outside AWS) costs money. Uploading is free. If you're moving terabytes around, do the math first.

For most Replit projects, you'll spend more on coffee than on S3. Go build something awesome!

-Sethers


```json
{"@context": "https://schema.org", "@type": "BlogPosting", "headline": "Replit Object Storage with AWS S3: Python and TypeScript Setup", "datePublished": "2025-12-12T04:02:33", "dateModified": "2026-09-17T19:30:52", "image": {"@type": "ImageObject", "url": "https://www.sethserver.com/static/images/ai-posts/connecting-your-replit-app-to-aws-s3-social.jpg", "width": 400, "height": 400}, "url": "https://www.sethserver.com/programming/connecting-your-replit-app-to-aws-s3.html", "author": {"@type": "Person", "name": "Seth Black", "url": "https://www.linkedin.com/in/seth-black-tx"}, "publisher": {"@type": "Organization", "name": "Sethserver", "logo": {"@type": "ImageObject", "url": "https://www.sethserver.com/static/images/PaperAirplane512.png"}}, "description": "Use AWS S3 as object storage for a Replit app: create the bucket, an IAM user with tight permissions, and Replit Secrets, then upload and download files with boto3 or the AWS SDK v3.", "articleSection": "Technology"}
{"@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [{"@type": "ListItem", "position": 1, "name": "sethserver", "item": "https://www.sethserver.com/"}, {"@type": "ListItem", "position": 2, "name": "Programming", "item": "https://www.sethserver.com/programming/"}, {"@type": "ListItem", "position": 3, "name": "Replit Object Storage with AWS S3: Python and TypeScript Setup", "item": "https://www.sethserver.com/programming/connecting-your-replit-app-to-aws-s3.html"}]}
```
