Skip to content

feat: support API access in local server #108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,42 @@ for await (const entry of store.list({ paginate: true })) {
console.log(blobs)
```

## Server API reference

We provide a Node.js server that implements the Netlify Blobs server interface backed by the local filesystem. This is
useful if you want to write automated tests that involve the Netlify Blobs API without interacting with a live store.

The `BlobsServer` export lets you construct and initialize a server. You can then use its address to initialize a store.

```ts
import { BlobsServer, getStore } from '@netlify/blobs'

// Choose any token for protecting your local server from
// extraneous requests
const token = 'some-token'

// Create a server by providing a local directory where all
// blobs and metadata should be persisted
const server = new BlobsServer({
directory: '/path/to/blobs/directory',
port: 1234,
token,
})

await server.start()

// Get a store and provide the address of the local server
const store = getStore({
edgeURL: 'http://localhost:1234',
name: 'my-store',
token,
})

await store.set('my-key', 'This is a local blob')

console.log(await store.get('my-key'))
```

## Contributing

Contributions are welcome! If you encounter any issues or have suggestions for improvements, please open an issue or
Expand Down
2 changes: 1 addition & 1 deletion src/list.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ const siteID = '9a003659-aaaa-0000-aaaa-63d3720d8621'
const storeName = 'mystore'
const apiToken = 'some token'
const edgeToken = 'some other token'
const edgeURL = 'https://cloudfront.url'
const edgeURL = 'https://edge.netlify'

describe('list', () => {
describe('With API credentials', () => {
Expand Down
2 changes: 1 addition & 1 deletion src/main.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ const value = 'some value'
const apiToken = 'some token'
const signedURL = 'https://signed.url/123456789'
const edgeToken = 'some other token'
const edgeURL = 'https://cloudfront.url'
const edgeURL = 'https://edge.netlify'

describe('get', () => {
describe('With API credentials', () => {
Expand Down
41 changes: 41 additions & 0 deletions src/server.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -215,3 +215,44 @@ test('Lists entries', async () => {

expect(parachutesSongs2.directories).toEqual([])
})

test('Supports the API access interface', async () => {
const directory = await tmp.dir()
const server = new BlobsServer({
directory: directory.path,
token,
})
const { port } = await server.start()
const blobs = getStore({
apiURL: `http://localhost:${port}`,
name: 'mystore',
token,
siteID,
})
const metadata = {
features: {
blobs: true,
functions: true,
},
name: 'Netlify',
}

await blobs.set('simple-key', 'value 1')
expect(await blobs.get('simple-key')).toBe('value 1')

await blobs.set('simple-key', 'value 2', { metadata })
expect(await blobs.get('simple-key')).toBe('value 2')

await blobs.set('parent/child', 'value 3')
expect(await blobs.get('parent/child')).toBe('value 3')
expect(await blobs.get('parent')).toBe(null)

const entry = await blobs.getWithMetadata('simple-key')
expect(entry?.metadata).toEqual(metadata)

await blobs.delete('simple-key')
expect(await blobs.get('simple-key')).toBe(null)

await server.stop()
await fs.rm(directory.path, { force: true, recursive: true })
})
38 changes: 35 additions & 3 deletions src/server.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { createHmac } from 'node:crypto'
import { createReadStream, createWriteStream, promises as fs } from 'node:fs'
import http from 'node:http'
import { tmpdir } from 'node:os'
Expand All @@ -7,6 +8,9 @@ import { ListResponse } from './backend/list.ts'
import { decodeMetadata, encodeMetadata, METADATA_HEADER_INTERNAL } from './metadata.ts'
import { isNodeError, Logger } from './util.ts'

const API_URL_PATH = /\/api\/v1\/sites\/(?<site_id>[^/]+)\/blobs\/?(?<key>[^?]*)/
const DEFAULT_STORE = 'production'

interface BlobsServerOptions {
/**
* Whether debug-level information should be logged, such as internal errors
Expand Down Expand Up @@ -44,6 +48,7 @@ export class BlobsServer {
private port: number
private server?: http.Server
private token?: string
private tokenHash: string

constructor({ debug, directory, logger, port, token }: BlobsServerOptions) {
this.address = ''
Expand All @@ -52,6 +57,9 @@ export class BlobsServer {
this.logger = logger ?? console.log
this.port = port || 0
this.token = token
this.tokenHash = createHmac('sha256', Math.random.toString())
.update(token ?? Math.random.toString())
.digest('hex')
}

logDebug(...message: unknown[]) {
Expand Down Expand Up @@ -222,10 +230,23 @@ export class BlobsServer {
}

handleRequest(req: http.IncomingMessage, res: http.ServerResponse) {
if (!this.validateAccess(req)) {
if (!req.url || !this.validateAccess(req)) {
return this.sendResponse(req, res, 403)
}

const apiURLMatch = req.url.match(API_URL_PATH)

// If this matches an API URL, return a signed URL.
if (apiURLMatch) {
const fullURL = new URL(req.url, this.address)
const storeName = fullURL.searchParams.get('context') ?? DEFAULT_STORE
const key = apiURLMatch.groups?.key as string
const siteID = apiURLMatch.groups?.site_id as string
const url = `${this.address}/${siteID}/${storeName}/${key}?signature=${this.tokenHash}`

return this.sendResponse(req, res, 200, JSON.stringify({ url }))
}

switch (req.method) {
case 'DELETE':
return this.delete(req, res)
Expand Down Expand Up @@ -294,11 +315,22 @@ export class BlobsServer {
const { authorization = '' } = req.headers
const parts = authorization.split(' ')

if (parts.length !== 2 || parts[0].toLowerCase() !== 'bearer') {
if (parts.length === 2 || (parts[0].toLowerCase() === 'bearer' && parts[1] === this.token)) {
return true
}

if (!req.url) {
return false
}

return parts[1] === this.token
const url = new URL(req.url, this.address)
const signature = url.searchParams.get('signature')

if (signature === this.tokenHash) {
return true
}

return false
}

/**
Expand Down