S3Mock / server
AGENTS.md1.2k starsChanged 3 months agoApache-2.0View on GitHub
Inherits all conventions from the root AGENTS.md. Below are module-specific additions only. Core S3Mock server implementation. s3 and vectors are independent bounded contexts — neither may depend on the other (enforced by ArchitectureTest). Adding S3 operation: Follow DTO → Store → Service → Controller → IT: Each store uses a ConcurrentHashMap keyed by entity identity to hold one plain Any() lock object per entity. All reads and writes of metadata files must hold the corresponding lock. Pattern for adding a…
We split this AGENTS.md into the rules it gives an agent
4 rules. About 0 are common, 2 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 40
Adding S3 operation: Follow DTO → Store → Service → Controller → IT:
- Very specific: line 71
Never skip the lock for reads — getBucketMetadata and getS3ObjectMetadata are also synchronized.
- Distinctive: line 72
Never acquire more than one lock in a single call path — there is no established ordering, so taking two locks risks deadlock.
- Very specific: line 73
Do not introduce ReentrantLock, ReadWriteLock, or other lock types — the existing synchronized/Any() pattern is intentional and consistent …
≈ “Common” is an estimate: at least 1 in 5 of the 36,368 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Agent Context for S3Mock Server Module
Inherits all conventions from the root AGENTS.md. Below are module-specific additions only.
Core S3Mock server implementation.
Structure
server/src/main/kotlin/com/adobe/testing/s3mock/
├── S3MockApplication.kt # Spring Boot entry
├── S3MockConfiguration.kt # Top-level config
├── S3MockProperties.kt # Properties binding
├── common/ # Shared leaf package (s3 and vectors may both depend on it)
│ ├── StripedLocks.kt
│ └── AwsHttpHeaders.kt # AWS-specific header constants
├── s3/ # Core S3 API bounded context
│ ├── S3Exception.kt
│ ├── controller/
│ │ ├── *Controller.kt # REST endpoints
│ │ ├── ControllerConfiguration.kt # Controller beans + exception handlers
│ │ └── ControllerProperties.kt
│ ├── dto/ # XML/JSON models (*Result, request/response)
│ ├── model/ # Persisted metadata (BucketMetadata, S3ObjectMetadata, ...)
│ ├── service/
│ │ ├── *Service.kt # Business logic
│ │ └── ServiceConfiguration.kt # Service beans (used in @SpringBootTest)
│ └── store/
│ ├── *Store.kt # Persistence
│ ├── StoreConfiguration.kt # Store beans (used in @SpringBootTest)
│ └── StoreProperties.kt
└── vectors/ # S3 Vectors API bounded context (separate ports, JSON wire format)
├── controller/ service/ store/ dto/s3 and vectors are independent bounded contexts — neither may depend on the other (enforced by ArchitectureTest).
Implementation Flow
Adding S3 operation: Follow DTO → Store → Service → Controller → IT:
- DTO (
s3/dto/): Data classes with Jackson annotations — see rootAGENTS.md§ XML Serialization for the correcttools.jacksonannotations and namespace. Verify element names against AWS S3 API docs. - Store (
s3/store/): Filesystem path resolution, binary storage, metadata JSON. Key classes:BucketStore,ObjectStore,BucketMetadata,S3ObjectMetadata. Acquire the appropriate lock (see Locking section below). - Service (
s3/service/): Validation, store coordination. ThrowS3Exceptionconstants (e.g.,S3Exception.NO_SUCH_BUCKET) — see docs/SPRING.md for exception handling rules. - Controller (
s3/controller/): HTTP mapping only — delegate all logic to services. Controllers never catch exceptions. - Integration test (
integration-tests/): Real AWS SDK v2 against the Docker container — see integration-tests/AGENTS.md. Runmake integration-teststo verify XML serialization against the AWS S3 API. - Update docs:
CHANGELOG.md(user-facing entry) and rootAGENTS.mdConfiguration table if new properties are added.
Locking
Each store uses a ConcurrentHashMap keyed by entity identity to hold one plain Any() lock object per entity. All reads and writes of metadata files must hold the corresponding lock.
| Store | Lock key type | Lock map field | Where lock is registered |
BucketStore | String (bucket name) | lockStore | createBucket / loadBuckets via lockStore.putIfAbsent(bucketName, Any()) |
ObjectStore | UUID (object ID) | lockStore | Before first write via lockStore.putIfAbsent(id, Any()) |
MultipartStore | UUID (upload ID) | lockStore | On upload creation via lockStore.putIfAbsent(uploadId, Any()) |
Pattern for adding a store method that reads or writes metadata:
// Register lock lazily (writes only — reads rely on the lock already existing)
lockStore.putIfAbsent(id, Any())
// Acquire lock
synchronized(lockStore[id]!!) {
// read or write metadata here
}Rules:
- Never skip the lock for reads —
getBucketMetadataandgetS3ObjectMetadataare also synchronized. - Never acquire more than one lock in a single call path — there is no established ordering, so taking two locks risks deadlock.
- Do not introduce
ReentrantLock,ReadWriteLock, or other lock types — the existingsynchronized/Any()pattern is intentional and consistent throughout all stores.
Storage Schema
Filesystem layout:
<root>/<bucket>/bucketMetadata.json
<root>/<bucket>/<uuid>/binaryData
<root>/<bucket>/<uuid>/objectMetadata.json
<root>/<bucket>/<uuid>/<version-id>-binaryData # versioning
<root>/<bucket>/<uuid>/<version-id>-objectMetadata.json # versioning
<root>/<bucket>/multiparts/<upload-id>/multipartMetadata.json
<root>/<bucket>/multiparts/<upload-id>/<part-number>.part
<root>/<bucket>/multiparts/<upload-id>/<part-number>.partmeta.json # per-part checksum + sizebucketMetadata.json fields (BucketMetadata):
| Field | Type | Notes |
name | String | Bucket name |
creationDate | String | ISO-8601 timestamp |
bucketRegion | String | AWS region string |
objects | Map<String, UUID> | key → object UUID mapping |
versioningConfiguration | VersioningConfiguration? | null until versioning is configured |
objectLockConfiguration | ObjectLockConfiguration? | null until object lock is enabled |
bucketLifecycleConfiguration | BucketLifecycleConfiguration? | null until lifecycle rules are set |
objectOwnership | ObjectOwnership? | null until ownership is set |
bucketInfo | BucketInfo? | bucket type/data-redundancy info |
locationInfo | LocationInfo? | bucket location info |
path | Path | filesystem path to the bucket folder (serialized, but not portable across hosts or filesystem layouts) |
objectMetadata.json fields (S3ObjectMetadata):
| Field | Type | Notes |
id | UUID | object identity (matches the folder name) |
key | String | S3 object key |
size | String | content length as string |
contentType | String? | MIME type |
etag | String? | ETag value |
modificationDate | String | formatted date string |
lastModified | Long | epoch millis |
dataPath | Path | path to the binaryData file |
userMetadata | Map<String, String>? | x-amz-meta-* headers |
storeHeaders | Map<String, String>? | headers persisted verbatim (e.g. Content-Encoding) |
encryptionHeaders | Map<String, String>? | SSE headers |
tags | List<Tag>? | object tags |
checksumAlgorithm | ChecksumAlgorithm? | CRC32 / SHA-256 / etc. |
checksum | String? | computed checksum value |
checksumType | ChecksumType? | FULL_OBJECT or COMPOSITE |
storageClass | StorageClass? | STANDARD, GLACIER, etc. |
owner | Owner | object owner |
legalHold | LegalHold? | WORM legal hold status |
retention | Retention? | WORM retention mode + until-date |
policy | AccessControlPolicy? | ACL policy |
versionId | String? | non-null when versioning is enabled |
deleteMarker | Boolean | true for versioned delete markers |
parts | List<ObjectPart>? | per-part metadata for multipart-completed objects; null for single-PUT objects; populated at CompleteMultipartUpload from .partmeta.json sidecar files |
<part-number>.partmeta.json fields (PartMetadata) — written alongside each .part file during UploadPart; deleted after CompleteMultipartUpload:
| Field | Type | Notes |
partNumber | Int | S3 part number (1-based) |
etag | String? | ETag of the part |
size | Long | part size in bytes |
lastModified | Long | epoch millis when the part was uploaded |
checksum | String? | base64-encoded checksum value, or null if not provided |
checksumAlgorithm | ChecksumAlgorithm? | algorithm used, or null |
Testing
See docs/TESTING.md for the full strategy. Service and store tests use @SpringBootTest with @MockitoBean; controller tests use @WebMvcTest with @MockitoBean and BaseControllerTest. Always extend the appropriate base class (ServiceTestBase, StoreTestBase, BaseControllerTest).
Configuration
Three @ConfigurationProperties classes bind environment variables to typed properties:
StoreProperties(com.adobe.testing.s3mock.store.*) — storage root, buckets, KMS, regionControllerProperties(com.adobe.testing.s3mock.controller.*) — context pathS3MockProperties(com.adobe.testing.s3mock.*) — top-level settings
When adding, renaming, or removing a property, you must also update the testsupport modules that expose it to users:
testsupport/testcontainers/— add/update awithX()method andPROP_Xenv var constant inS3MockContainer(uppercase Spring key, replace.with_)testsupport/common/— add/update awithX()method andPROP_Xconstant inS3MockStarter(Spring key form)AGENTS.md(root) — update the Configuration section env var table if the property is user-facingREADME.md— update the configuration table if the property is user-facing
Trending
More talk lately.about the whole repository
Who's talking about adobe/S3Mock
- shishirsharma/MyS3Browser“Feature Request: Allow Option of Endpoint-URL ”GitHub · May 28, 2020 · 2 reactions
- Stack Overflow“how to make local machine to assume IAM role”Stack Overflow · Apr 13, 2019 · 2 votes
- gardener/etcd-backup-restore“Find alternative to LocalStack for S3 emulation in e2e tests”GitHub · Mar 25, 2026 · 1 reactions
- swisstopo/swissgeol-assets-suite“Minio ist deprecated”GitHub · Feb 17, 2026 · 1 reactions
- aws/aws-sdk-kotlin“S3Client error java.lang.NoClassDefFoundError: kotlin/coroutines/jvm/internal/SpillingKt with 1.5.0+”GitHub · Jul 25, 2025 · 1 reactions
The biggest 5 of 50 mentions linking to the repository that our crawl found. Each line is their own headline.
See all 50
Talked about on GitHub, Stack Overflow and Hacker News (50 times).
On GitHub · 46
- May 28, 2020Feature Request: Allow Option of Endpoint-URL — shishirsharma/MyS3Browser2 reactions · 3 comments
- Mar 25, 2026Find alternative to LocalStack for S3 emulation in e2e tests — gardener/etcd-backup-restore1 reaction · 2 comments
- Feb 17, 2026Minio ist deprecated — swisstopo/swissgeol-assets-suite1 reaction
- Jul 25, 2025S3Client error java.lang.NoClassDefFoundError: kotlin/coroutines/jvm/internal/SpillingKt with 1.5.0+ — aws/aws-sdk-kotlin1 reaction · 8 comments
- Mar 27, 2024use Adobe s3mock library — togglz/togglz1 reaction · 4 comments
41 more on GitHub
- Feb 14, 2024Migration d'Azure vers S3 — SocialGouv/cdtn-admin1 reaction · 1 comment
- Dec 5, 2022S3 Endpoint > Not able to use custom endpoint — aws/aws-sdk-php1 reaction · 9 comments
- Jun 3, 2022Use buildx, set tag to current version fails if it contains `-SNAPSHOT` — fabric8io/docker-maven-plugin1 reaction · 2 comments
- Jun 29, 2020Removed pinned version of adobe/s3mock docker image — overleaf/track-changes1 reaction · 1 comment
- Oct 9, 2026chore(storage): qualifier un backend S3 maintenu pour local/CI après archivage MinIO — NeooeN45/Quintessences0 reactions · 6 comments
- Sep 12, 2026docs: 확장 가능한 파일 저장 설계 및 구현 로드맵 추가 — dev-inho/backend-init0 reactions
- Aug 27, 2026Pin test container images, floating tags break the build when upstream changes — geosolutions-it/imageio-ext0 reactions
- Apr 26, 2026Add s3mock container — testcontainers/testcontainers-rs-modules-community0 reactions · 2 comments
- Apr 15, 2026[BUGFIX] Corriger l'initialisation des buckets S3 dans l'image docker — 1024pix/pix0 reactions · 2 comments
- Apr 8, 2026Update to latest S3Mock, which uses the updated `testcontainers` for Docker v29+ — guardian/etag-caching0 reactions
- Feb 20, 2026Update `testcontainers` to a version that works with Docker v29 — guardian/etag-caching0 reactions
- Jan 29, 2026Read-only image configuration for production-like testing — CMSgov/bluebutton-web-serversays it tested it0 reactions · 1 comment
- Jan 13, 2026Support for S3 data — fractal-analytics-platform/fractal-server0 reactions · 12 comments
- Dec 5, 2025Allow Specifying working directory on a per-invocation basis — woodpecker-ci/woodpecker0 reactions · 5 comments
- Oct 30, 2025Add S3Mock to implementation matrix — nikeee/lean-s30 reactions
- Jul 27, 2025Setup S3Mock in the compose setup — tchoutri/masna30 reactions
- Oct 8, 2024Upgrade Adobe s3mock to version 3.11.0 — apache/polaris0 reactions
- Aug 24, 2024DBZ-8169 Upgrade Adobe s3mock to 3.10.0 — debezium/debezium0 reactions · 1 comment
- Apr 18, 2024Publish large message payloads to S3 — sedaprotocol/seda-chain0 reactions · 1 comment
- Apr 7, 2024Document how to use the Amazon S3 Media module with a local storage emulator, implement support for it if necessary — OrchardCMS/OrchardCore0 reactions · 1 comment
- Mar 7, 2024Enhance local dev env — stackernews/stacker.news0 reactions · 6 comments
- Mar 2, 2024s3mock 3.6.0 — apache/pinot0 reactions · 3 comments
- Jan 17, 2024Add a dockerized S3 instance, for testing — duolicious/duolicioussays it tested it0 reactions
- Nov 25, 2023Retrieve, store and display asset metadata — royllo/explorer0 reactions
- Aug 31, 2023[Epic] Provide local filesystem-backed mock S3 service — cryostatio/cryostat0 reactions · 5 comments
- Jul 19, 2023Change suggested to add Dependabot config for a subfolder even though that exists already — step-security/secure-repo0 reactions · 1 comment
- Jul 13, 2023Mock S3 for testcase server — skkuding/codedang0 reactions
- May 16, 2023Add integration tests with an S3 API mock — jgroups-extras/jgroups-aws0 reactions · 1 comment
- Aug 10, 2022Multi-platform buildx: goal "push" fails with "Cannot extract archive" — fabric8io/docker-maven-plugin0 reactions · 4 comments
- Jun 1, 2022S3 storage ITs — artipie/asto0 reactions
- Mar 3, 2022File creation year is inaccurate by default — mathieucarbou/license-maven-plugin0 reactions · 1 comment
- Mar 10, 2021CommonPrefixes not parsed correctly if it contains multiple Prefix tags — boto/botocore0 reactions · 6 comments
- Feb 3, 2021Try switching s3 mocks — OCFL/ocfl-java0 reactions · 1 comment
- Feb 3, 2021Configure publication to Maven Central using Github actions — moditect/layrry0 reactions · 5 comments
- Jun 24, 2020Add Integration tests for AWS as part of the CI pipeline. — aslotte/MLOps.NET0 reactions
- Feb 25, 2020MultiPartUpload response content type — gaul/s3proxy0 reactions · 5 comments
- Mar 8, 2019Create tests for testing the S3.go file — smithoss/gonymizersays it tested it0 reactions
- Jan 9, 2019[CI/Python] Have automatized tests for S3 — apache/arrow0 reactions · 3 comments
- Jan 6, 2019Investigate using mock services for tests — scottwernervt/cloudstorage0 reactions
- Nov 22, 2018[research] Investigate using Adobe's S3 mock — franklinchou/csv-2-cassandra0 reactions
- Jul 11, 2018track adobe/s3mock update, since it is not using the latest version of github repo — apache/pulsar0 reactions · 1 comment
On Stack Overflow · 3
- Apr 13, 2019how to make local machine to assume IAM role2 votes
- Apr 12, 2024how to mock s3 using pytest? Unittest script calls actual function instead of mocking it?0 votes
- Oct 10, 2024Unable to replace python function definition with mock and unitest-1 votes
On Hacker News · 1
- Sep 10, 2020Javac could leak higher JDK bytecode1 point
Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.
Of the AGENTS.md files we read, the 28,693 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.
In this repository10 other files it gives its agents.
- AGENTS.mdAGENTS.md
- integration-tests/AGENTS.mdAGENTS.md
- testsupport/AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- documentSkill
- implementSkill
- lintSkill
- refactorSkill
- reviewSkill
- testSkill
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

