Skip to content
BUILD FOUNDRY

Engineering

Signing S3 requests without the SDK

Two operations did not justify the AWS SDK in a serverless bundle, so we signed them by hand. How SigV4 works, the four traps that cost the most time, and when the SDK is still the right answer.

Published
Reading time
About 11 minutes
Written by
The engineering team. There is no marketing department here to write it for us.

This site stores one thing: a brief attached to a contact-form submission. It needs two operations against object storage — put an object, and produce a temporary signed URL so the file opens from an email a week later. That is the entire surface.

The obvious answer is `@aws-sdk/client-s3` plus `@aws-sdk/s3-request-presigner`. We did not take it, and the reasoning is narrow enough to be worth stating precisely, because for most people the SDK is still correct.

Why not the SDK, for this specific case

The SDK is modular, but a client is still a client: it arrives with a middleware stack, a credential-provider chain, a retry strategy, XML serialisation and a signer, spread across a dozen packages. All of that exists to solve real problems — refreshable credentials, instance metadata, multipart uploads, adaptive retry — and this application has none of them. It has two static keys in environment variables and two calls.

In a serverless deployment the cost of a dependency is not only install time. It is bundle size, which is cold-start time, on a function that runs when somebody submits a form and then not again for hours. We are not going to quote you a megabyte figure, because it depends on your bundler, your tree-shaking and the SDK version, and a number from someone else’s blog post is not evidence about your build. Measure it in your own bundle. Ours was large enough, against a surface this small, that the trade went the other way.

The other half of the trade: Node has HMAC-SHA256 and SHA-256 in `node:crypto`, and the storage is Cloudflare R2, which speaks the S3 API. Signature Version 4 is a published, stable specification. The whole implementation is about eighty lines with no dependencies at all.

How SigV4 works

Four steps. The idea is that both sides independently build the same exact string from the request, hash it, and sign it with a key derived from the secret — so the signature covers the method, the path, the query, the chosen headers and the body, and any change to any of them invalidates it.

Step one, the canonical request. A deterministic text rendering of the request, newline-separated, in a fixed order.

The canonical request
HTTPMethod
CanonicalURI
CanonicalQueryString
CanonicalHeaders
SignedHeaders
HashedPayload

CanonicalHeaders is one `name:value` pair per line, lowercased, values trimmed, sorted by name, each line terminated with a newline — so the block ends with a blank line before SignedHeaders. SignedHeaders is those same names, lowercase, sorted, joined with semicolons. HashedPayload is the hex SHA-256 of the body, or the literal string UNSIGNED-PAYLOAD.

Step two, the string to sign. The canonical request gets hashed and wrapped in the algorithm, the timestamp and the credential scope.

The string to sign
AWS4-HMAC-SHA256
20260825T104512Z
20260825/auto/s3/aws4_request
hex(sha256(canonicalRequest))

The scope line is date, region, service, and the constant aws4_request. It binds a signature to one day, one region and one service, which is why a leaked signature is bounded rather than permanent.

Step three, the signing key. Four chained HMACs, each using the previous digest as the key. The point of the chain is that the derived key is scoped the same way the credential is, so it can be distributed to a signer without handing over the root secret.

src/lib/storage.ts — key derivation
const hmac = (key: string | Buffer, value: string) =>
  createHmac("sha256", key).update(value).digest();

function signingKey(dateStamp: string): Buffer {
  const kDate = hmac(`AWS4${storageConfig.secretAccessKey}`, dateStamp);
  const kRegion = hmac(kDate, storageConfig.region);
  const kService = hmac(kRegion, SERVICE);
  return hmac(kService, "aws4_request");
}

Note that the first step prefixes the literal AWS4 to the secret, and that every intermediate value is raw bytes rather than hex. Hex-encoding an intermediate digest is the single most common way to get a signature that is wrong in a way you cannot see.

Step four, sign the string to sign with that key and hex-encode the result. For a request, it goes in an Authorization header alongside the credential and the signed-header list. For a presigned URL, it goes in the query string as X-Amz-Signature.

The traps

The specification is clear. What costs time is that every mistake produces the same response — SignatureDoesNotMatch — which reads like a credentials problem and is almost never a credentials problem. Four of them accounted for most of the debugging.

1. Path encoding, where slashes are not slashes

The canonical URI is the path with each segment URI-encoded, but the slashes between segments left alone. Encoding the slashes gives you a signature over a different path than the one you are requesting. So does not encoding the segments.

There is a second, subtler half. AWS specifies RFC 3986 encoding, where the unreserved set is exactly A–Z, a–z, 0–9, hyphen, underscore, period and tilde. Everything else is percent-encoded. JavaScript’s `encodeURIComponent` does not agree: it leaves `!`, `'`, `(`, `)` and `*` literal. Those characters are entirely legal in an object key, and a key containing one signs correctly right up until it does not.

src/lib/storage.ts
function encodeKey(key: string): string {
  return key
    .split("/")
    .map((segment) =>
      encodeURIComponent(segment).replace(
        /[!'()*]/g,
        (char) => `%${char.charCodeAt(0).toString(16).toUpperCase()}`,
      ),
    )
    .join("/");
}

Split, encode, patch the five stragglers, rejoin. The same function produces the public URL, so the signed path and the served path cannot drift apart.

2. Signed headers must match what is actually sent

SignedHeaders is a promise about the request. If you list content-type, the request must carry content-type, byte for byte the value you signed. A header that is signed and not sent fails. A signed value that differs by a space fails.

The practical rule is to sign only headers you set yourself, and to build the header object once and use it for both the signature and the request rather than assembling it twice.

src/lib/storage.ts — one object, two uses
const headers: Record<string, string> = {
  host,
  "content-type": contentType,
  "x-amz-content-sha256": payloadHash,
  "x-amz-date": amzDate,
};

const signedHeaders = Object.keys(headers).sort().join(";");
const canonicalHeaders = Object.keys(headers)
  .sort()
  .map((name) => `${name}:${headers[name].trim()}\n`)
  .join("");

const response = await fetch(`${storageConfig.endpoint}${canonicalUri}`, {
  method: "PUT",
  headers: { ...headers, Authorization: authorization },
  body: new Uint8Array(options.body),
  cache: "no-store",
});

There is a security consequence hiding in that content-type. The value comes from the browser, which means it is attacker-controlled, and it gets signed into the request. A malformed value makes `fetch` throw before the request leaves — and since the caller treats an upload failure as non-fatal, a hostile or merely broken MIME type would silently cost a client their attachment. It is validated against a pattern before it is signed, and falls back to application/octet-stream.

3. UNSIGNED-PAYLOAD on the presigned GET

The hashed-payload slot in the canonical request must contain something. For the upload we have the body in memory, so it is the real SHA-256 and the signature covers the bytes. For a presigned GET there is no body at all, and more importantly the person who eventually opens the link is not the person who signed it and could not compute a hash if they wanted to. The literal string UNSIGNED-PAYLOAD goes in that slot instead, and both sides agree on the substitution.

src/lib/storage.ts — the presigned GET canonical request
const canonicalRequest = [
  "GET",
  canonicalUri,
  query.toString(),
  `host:${host}\n`,
  "host",
  UNSIGNED_PAYLOAD,
].join("\n");

Only host is signed here, which is the point of query-string signing: the URL has to work in a mail client that will send no custom headers whatsoever. SigV4 allows a presigned URL to live for up to seven days, and the temptation is to use all of it. We use 48 hours. The URL goes in an email, and email is forwarded, quoted and archived — a week-long window means anyone who later reaches that mailbox has unauthenticated access to a client's brief, with no revocation and no audit trail. The permanent record is the object key, so an expired link can always be re-signed.

4. URLSearchParams is not RFC 3986

This one is a trap rather than a bug we hit, and it is the kind worth writing down because the code works today and would break on a change that looks unrelated.

`URLSearchParams.toString()` produces application/x-www-form-urlencoded output, which is a form serialisation, not a URI serialisation. Three differences matter against what AWS expects. A space becomes `+` rather than `%20`. An asterisk is left literal rather than encoded to `%2A`. A tilde is encoded to `%7E` rather than left literal.

None of our five query values contains a space, an asterisk or a tilde. The credential value contains slashes, and those are correctly encoded to `%2F` by both schemes, which is the one that would have bitten immediately. So the current code is correct — and it is correct by luck about the input, not by construction.

src/lib/storage.ts — sorted by key, byte order
const query = new URLSearchParams({
  "X-Amz-Algorithm": ALGORITHM,
  "X-Amz-Credential": `${storageConfig.accessKeyId}/${scope}`,
  "X-Amz-Date": amzDate,
  "X-Amz-Expires": String(expiresInSeconds),
  "X-Amz-SignedHeaders": "host",
});
query.sort();

The sort is not cosmetic: the canonical query string must be ordered by parameter name in byte order, and the string that is signed has to be the identical string that is sent. Anyone adding a parameter here — a response-content-disposition override, say, which very much can contain spaces — needs to replace URLSearchParams with an explicit RFC 3986 encoder rather than assume it will continue to work.

How it was verified

Honestly: against AWS’s published reference vectors before it went anywhere near a bucket. The signing documentation includes a worked example with a fixed secret, a fixed timestamp and the expected intermediate values — the canonical request, its hash, the string to sign, the derived key and the final signature. Every stage was compared against those.

That order matters more than it sounds. A live bucket gives you one bit of information — accepted or SignatureDoesNotMatch — for an implementation with five places to go wrong. The published vectors tell you which stage diverged, which turns an afternoon of guessing into a five-minute comparison. If you implement this yourself, do that first.

Two operational details worth knowing before you debug the wrong thing. Requests carry a timestamp and are rejected outside a fifteen-minute window, so a container with a drifting clock produces authentication failures that look exactly like bad keys. And R2 ignores the region entirely, but SigV4 still requires one in the scope — it is set to auto here, and the only requirement is that the value used in the credential scope and the value used to derive the key are the same.

When the SDK is the right answer

Almost always. This trade only works because the surface is two operations, both are stable parts of the S3 API that have not changed in years, the credentials are static, and the failure mode is acceptable — a lost attachment on an enquiry that still reaches us with the filename named in the email.

Use the SDK the moment any of that stops being true: multipart uploads, listing, lifecycle or bucket configuration, retry and backoff you would otherwise write yourself, checksums, credentials that refresh, assumed roles or instance metadata, or anything where you need to keep pace with a service that is still evolving. Hand-signing is correct for a narrow, stable, well-understood surface. It is a liability the moment the surface starts growing, and the honest failure mode of this approach is a future engineer adding a third operation to a signer nobody has looked at in a year.

The version described here is in `src/lib/storage.ts` in this repository, comments and all. This kind of trade — a dependency against a maintenance surface — is one of the things we look at when we read somebody else’s system. How we read an inherited codebase →

Send us a URL and we will write up what is healthy, what is at risk and what we would fix first — in the same plain terms as everything above. No cost, and no obligation to continue afterwards.