getAmplifyDataClientConfig() fails to check Amplify data variables when API name is defined

[lengieng]

Environment information

System:
  OS: Windows 11 10.0.26100
  CPU: (12) x64 13th Gen Intel(R) Core(TM) i5-1335U
  Memory: 2.08 GB / 15.72 GB
Binaries:
  Node: 22.2.0 - C:\Program Files\nodejs\node.EXE
  Yarn: 1.22.22 - ~\AppData\Roaming\npm\yarn.CMD
  npm: 10.9.1 - C:\Program Files\nodejs\npm.CMD
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/auth-construct: 1.5.1
  @aws-amplify/backend: 1.9.0
  @aws-amplify/backend-auth: 1.4.2
  @aws-amplify/backend-cli: 1.4.3
  @aws-amplify/backend-data: 1.2.2
  @aws-amplify/backend-deployer: 1.1.11
  @aws-amplify/backend-function: 1.9.0
  @aws-amplify/backend-output-schemas: 1.4.0
  @aws-amplify/backend-output-storage: 1.1.4
  @aws-amplify/backend-secret: 1.1.5
  @aws-amplify/backend-storage: 1.2.4
  @aws-amplify/cli-core: 1.2.1
  @aws-amplify/client-config: 1.5.3
  @aws-amplify/deployed-backend-client: 1.4.2
  @aws-amplify/form-generator: 1.0.3
  @aws-amplify/model-generator: 1.0.9
  @aws-amplify/platform-core: 1.3.0
  @aws-amplify/plugin-types: 1.6.0
  @aws-amplify/sandbox: 1.2.7
  @aws-amplify/schema-generator: 1.2.6
  aws-amplify: 6.10.2
  aws-cdk: 2.172.0
  aws-cdk-lib: 2.172.0
  typescript: 5.6.2
No AWS environment variables
No CDK environment variables

Describe the bug

The doc shows an example of how to use defineAuth and defineFunction to create Cognito post confirmation Lambda trigger. Another related section is grant Lambda function access to API.

When name is given in the defineData(), the generated env that start with the AMPLIFY_DATA prefix are replaced with the given name.

export const data = defineData({
  name: 'CoolName',
  schema,
  authorizationModes: {
    defaultAuthorizationMode: 'apiKey',
    apiKeyAuthorizationMode: {
      expiresInDays: 90,
    },
  },
})

Without name given (default):

AMPLIFY_DATA_GRAPHQL_ENDPOINT 
AMPLIFY_DATA_MODEL_INTROSPECTION_SCHEMA_BUCKET_NAME
AMPLIFY_DATA_MODEL_INTROSPECTION_SCHEMA_KEY

With name given:

COOL_NAME_GRAPHQL_ENDPOINT 
COOL_NAME_MODEL_INTROSPECTION_SCHEMA_BUCKET_NAME
COOL_NAME_MODEL_INTROSPECTION_SCHEMA_KEY

In this case, in the amplify/auth/post-confirmation/handler.ts, this line const { resourceConfig, libraryOptions } = await getAmplifyDataClientConfig(env) fails to generate the resourceConfig because the isDataClientEnv(env) used in the getAmplifyDataClientConfig() always returns false.

Code below taken from here in the recent pull commit 2224.

const isDataClientEnv = (env: unknown): env is DataClientEnv => {
  return (
    env !== null &&
    typeof env === 'object' &&
    'AMPLIFY_DATA_MODEL_INTROSPECTION_SCHEMA_BUCKET_NAME' in env && // <-- FAIL
    'AMPLIFY_DATA_MODEL_INTROSPECTION_SCHEMA_KEY' in env &&  // <-- FAIL
    'AWS_ACCESS_KEY_ID' in env &&
    'AWS_SECRET_ACCESS_KEY' in env &&
    'AWS_SESSION_TOKEN' in env &&
    'AWS_REGION' in env &&
    'AMPLIFY_DATA_GRAPHQL_ENDPOINT' in env && // <-- FAIL
    typeof env.AMPLIFY_DATA_MODEL_INTROSPECTION_SCHEMA_BUCKET_NAME ===
      'string' && // <-- FAIL
    typeof env.AMPLIFY_DATA_MODEL_INTROSPECTION_SCHEMA_KEY === 'string' && // <-- FAIL
    typeof env.AWS_ACCESS_KEY_ID === 'string' &&
    typeof env.AWS_SECRET_ACCESS_KEY === 'string' &&
    typeof env.AWS_SESSION_TOKEN === 'string' &&
    typeof env.AWS_REGION === 'string' &&
    typeof env.AMPLIFY_DATA_GRAPHQL_ENDPOINT === 'string' // <-- FAIL
  )
}

Reproduction steps

Follow the example in the doc with an additional step to add the name to defineData().

1. Give the API a name in defineData()

// amplify/data/resource.ts
import { type ClientSchema, a, defineData } from "@aws-amplify/backend";
import { postConfirmation } from "../auth/post-confirmation/resource";

const schema = a
  .schema({
    UserProfile: a
      .model({
        email: a.string(),
        profileOwner: a.string(),
      })
      .authorization((allow) => [
        allow.ownerDefinedIn("profileOwner"),
      ]),
  })
  .authorization((allow) => [allow.resource(postConfirmation)]);
export type Schema = ClientSchema<typeof schema>;

export const data = defineData({
  name: "CoolName", // Give the API a name
  schema,
  authorizationModes: {
    defaultAuthorizationMode: "apiKey",
    apiKeyAuthorizationMode: {
      expiresInDays: 30,
    },
  },
});

2. Create post-confirmation resource file amplify/auth/post-confirmation/resource.ts:

// amplify/auth/post-confirmation/resource.ts
import { defineFunction } from '@aws-amplify/backend';

export const postConfirmation = defineFunction({
  name: 'post-confirmation',
});

3. Create post-confirmation handler file amplify/auth/post-confirmation/handler.ts:

// amplify/auth/post-confirmation/handler.ts
import type { PostConfirmationTriggerHandler } from "aws-lambda";
import { type Schema } from "../../data/resource";
import { Amplify } from "aws-amplify";
import { generateClient } from "aws-amplify/data";
import { getAmplifyDataClientConfig } from '@aws-amplify/backend/function/runtime';
import { env } from "$amplify/env/post-confirmation";

const { resourceConfig, libraryOptions } = await getAmplifyDataClientConfig(
  env
);

Amplify.configure(resourceConfig, libraryOptions);

const client = generateClient<Schema>();

export const handler: PostConfirmationTriggerHandler = async (event) => {
  await client.models.UserProfile.create({
      email: event.request.userAttributes.email,
      profileOwner: `${event.request.userAttributes.sub}::${event.userName}`,
  });

  return event;
};

4. Set trigger in amplify/auth/resource.ts

// amplify/auth/resource.ts
import { defineAuth } from '@aws-amplify/backend';
import { postConfirmation } from './post-confirmation/resource';

export const auth = defineAuth({
  loginWith: {
    email: true,
  },
  triggers: {
    postConfirmation
  }
});

[stocaaro]

Hello @lengieng, Thank you for the detailed notes on this issue. I am working on a fix and will post updates as I make progress.

[stocaaro]

@lengieng, The fix for this was released with @aws-amplify/backend@1.11.0. Can you confirm that the feature is working for you with the latest version?

[dlamon1]

Not OP, I'm gettin the following error from 1.11.0

Argument of type 'ResourceConfig | { invalidType: "Some of the AWS environment variables needed to configure Amplify are missing."; }' is not assignable to parameter of type 'ResourcesConfig | LegacyConfig | AmplifyOutputs'.
  Type '{ invalidType: "Some of the AWS environment variables needed to configure Amplify are missing."; }' is not assignable to type 'ResourcesConfig | LegacyConfig | AmplifyOutputs'.ts(2345)
const resourceConfig: ResourceConfig | {
    invalidType: "Some of the AWS environment variables needed to configure Amplify are missing.";
}

import { type Schema } from '../../data/resource';
import { Amplify } from 'aws-amplify';
import { generateClient } from 'aws-amplify/data';
import { getAmplifyDataClientConfig } from '@aws-amplify/backend/function/runtime';
import { env } from '$amplify/env/listTables';

const { resourceConfig, libraryOptions } = await getAmplifyDataClientConfig(env);

Amplify.configure(resourceConfig, libraryOptions);

[stocaaro]

@dlamon1,

I just stepped through the docs guide with a newly create react app. I get the same error you're seeing when I attempt to use getAmplifyDataClientConfig configuration when I haven't added permission to my data schema to allow it to be accessed by my function.

import { functionWithDataAccess } from '../functions/data-access/resource';
const schema = a.schema({
...
}).authorization((a) => [a.resource(functionWithDataAccess)]);

If you've done this and you're still seeing the error, can you share your .amplify/generated/env/listTables.ts file contents?

Thanks!

[dlamon1]

Sure, though what I posted is most of that resouce.ts

data/resource.ts

import { type ClientSchema, a, defineData } from '@aws-amplify/backend';
import { HardwareSchema } from './hardware.schema';
import { ProjectSchema } from './project.schema';
import { ConnectorSchema } from './connector.schema';
import { ManufacturerSchema } from './manufacturer.schema';
// import { FunctionsSchema } from './functions.schema';
import { onAdmin } from '../functions/onAdmin/resource';
import { listTables } from '../functions/listTables/resource';
import { clearRowsInTable } from '../functions/clearRowsInTable/resource';

const schema = a
  .schema({
    ...HardwareSchema,
    ...ProjectSchema,
    ...ConnectorSchema,
    ...ManufacturerSchema,
    // ...FunctionsSchema
    listTables: a
      .query()
      .arguments({})
      .returns(a.string())
      .handler(a.handler.function(listTables))
      .authorization((allow) => allow.group('ADMINS'))
  })
  .authorization((allow) => [allow.resource(listTables)]);

export type Schema = ClientSchema<typeof schema>;

export const data = defineData({
  schema,
  authorizationModes: {
    defaultAuthorizationMode: 'userPool'
  }
});

functions/listTables/resource.ts

// import { DynamoDB } from '@aws-sdk/client-dynamodb';
import type { Handler } from 'aws-lambda';
import { AmplifyClient, ListBranchesCommand } from '@aws-sdk/client-amplify';
import type { S3Handler } from 'aws-lambda';
import { type Schema } from '../../data/resource';
import { Amplify } from 'aws-amplify';
import { generateClient } from 'aws-amplify/data';
import { getAmplifyDataClientConfig } from '@aws-amplify/backend/function/runtime';
import { env } from '$amplify/env/listTables';

const { resourceConfig, libraryOptions } = await getAmplifyDataClientConfig(env);

Amplify.configure(resourceConfig, libraryOptions);

const client = generateClient<Schema>();

// Initialize the Amplify client
export const handler: Handler = async () => {}

[stocaaro]

@dlamon1 ,

Interesting. Your code from functions/listTables/resource.ts looks like the content I would expect to see in functions/listTables/handler.ts.

The function resource.ts files usually look something like:

import { defineFunction } from '@aws-amplify/backend';

export const listTables = defineFunction({
  name: 'listTables',
});

In data/resource.ts you have import { listTables } from '../functions/listTables/resource';, but listTables isn't exported in the copied code. I would guess that this is in the file, but outside the context you provided.

If you are merging the function resource and handler definition into a single file like this, you'll need to separate them out. The resource.ts files compose the CDK constructs needed to synthesize your backend where the handler.ts files are built and the built artifacts are deployed to lambda.

When I merge these files like this, I get other import error Cannot find package '$amplify'. I'm not sure how this would build to have the error you're encountering.

[dlamon1]

You're very kind at calling out my obvious mistake, thank you, you're right.

import { defineFunction } from '@aws-amplify/backend';

export const listTables = defineFunction({
  name: 'listTables'
});

[dlamon1]

*To clarify, this was the the resource.ts file I had already implement when receiving this error.

[dlamon1]

The listTables.ts file

// This file is auto-generated by Amplify. Edits will be overwritten.
export const env = process.env as any;

[stocaaro]

@dlamon1, Looking at the content for .amplify/generated/env/listTables.ts helps. This is the content that gets written to the file right before the function is built. When the build succeeds the fully typed content is written. I haven't been able to replicate this in my example project. Would it be possible to provide an example project repo that is experiencing the issue? It would be helpful to get your package.json, tsconfig and any lint config files, but not as helpful as having a complete example of the issue.

[jserrano-alq]

I'm having this issue for a Angular project, its a fresh project following the Create a user profile record

data/resource.ts

const schema = a.schema({
  NLBUser: a.model({
    id: a.string(),
    ...
    profileOwner: a.string(),
    ...
  }).authorization((allow) => [
    allow.ownerDefinedIn("profileOwner")
  ]),
...
}).authorization((allow) => [allow.publicApiKey(), allow.resource(postConfirmation)]);

auth/resource.ts

import { defineAuth } from '@aws-amplify/backend';
import { postConfirmation } from './post-confirmation/resource';

/**
 * Define and configure your auth resource
 * @see https://docs.amplify.aws/gen2/build-a-backend/auth
 */
export const auth = defineAuth({
  loginWith: {
    email: true,
  },
  triggers: {
    postConfirmation
  }
});

auth/post-confirmation/resource.ts

import { defineFunction } from '@aws-amplify/backend';

export const postConfirmation = defineFunction({
    name: 'post-confirmation',
});

auth/post-confirmation/handler.ts

import type { PostConfirmationTriggerHandler } from "aws-lambda";
import { type Schema } from "../../data/resource";
import { Amplify } from "aws-amplify";
import { generateClient } from "aws-amplify/data";
import { getAmplifyDataClientConfig } from '@aws-amplify/backend-function/runtime';
import { env } from "$amplify/env/post-confirmation";

const { resourceConfig, libraryOptions } = await getAmplifyDataClientConfig(
  env
);

Amplify.configure(resourceConfig, libraryOptions);

const client = generateClient<Schema>();

export const handler: PostConfirmationTriggerHandler = async (event) => {
  await client.models.NLBUser.create({
    email: event.request.userAttributes.email,
    profileOwner `${event.request.userAttributes.sub}::${event.userName}`,
  });

  return event;
};

On the statement Amplify.configure(resourceConfig, libraryOptions); I get the error

Argument of type '{ invalidType: "Some of the AWS environment variables needed to configure Amplify are missing."; }' is not assignable to parameter of type 'ResourcesConfig | LegacyConfig | AmplifyOutputs'.

On the statement

const { resourceConfig, libraryOptions } = await getAmplifyDataClientConfig(
  env
);

Both resourceConfig and resourceConfig are returning

invalidType: "Some of the AWS environment variables needed to configure Amplify are missing.";

Where could I find those missing variables or how can i define them? Thanks!

One solution I found while on sandbox mode was doing

import outputs from '../amplify_outputs.json';

Amplify.configure(outputs);

But when deployed it gives the error that amplify_outputs.json not found and its because the file is not committed, I believe that its for good practice that it shouldn't be added, since its the way amplify is intended to work.

Sorry If im making some obvious mistake, new around here :)

[stocaaro]

Hi @jserrano-alq,

Thank you for all the details. I think I've gone through the same process, starting an Angular app and setting up amplify with an auth trigger to create a profile record.

I didn't encounter the error you're seeing, but I've capture my example app here.

Locally, I'm using node v20.18.0 (npm v10.8.2), which versions are you building with?

You're right that using '../amplify_outputs.json' directly doesn't work. Both sandbox and deploy don't construct the outputs file until after the cdk constructs have been synthesized. Locally, the sandbox will use the outputs file built by a prior run, but deploying only runs once, so will always encounter that error.

I don't see any obvious mistakes or differences and I appreciate your help in working to track down the difference that is causing this error in your environment.

[spyderbr6]

ugh my whole morning was thrown away trying to troubleshoot this. i'm glad there was a patch but now i'm in the same place jseranno-alq is after the patch... i'll try to troubleshoot more tonight and will see if i can leverage claude. will post findings if something obvious comes out of it.

[dryhurst]

in my case .env files are not being populated at all and therefore i get the above error previously reported Argument of type '{ invalidType: "Some of the AWS environment variables needed to configure Amplify are missing."; }' snipped

[stocaaro]

Hello @dryhurst and @spyderbr6,

I understand how frustrating it is when things don't work right. If you can provide either an example app repo that exhibits this problem or step by step instructions (including cli commands used and node version) to reproduce the problem, it would help accelerate our investigation. I've tried several approaches without running into the issue and will be investigating further today.

Prior to releasing getAmplifyDataClientConfig the recommended way to make api calls inside a function was to use the client.graphql to call queries generated by codegen.

If your blocked on this issue and are looking for an workaround to get unblocked while we investigate, here are the docs for postConfirmation that use client.graphql.

Run the command npx ampx sandbox to create the backend, then use the command below to generate GraphQL client code to call your data backend.

npx ampx generate graphql-client-code --out <path-to-post-confirmation-handler-dir>/graphql

Then, create the corresponding handler file, amplify/auth/post-confirmation/handler.ts, file with the following contents:

import type { PostConfirmationTriggerHandler } from "aws-lambda";
import { type Schema } from "../../data/resource";
import { Amplify } from "aws-amplify";
import { generateClient } from "aws-amplify/data";
import { env } from "$amplify/env/post-confirmation";
import { createUserProfile } from "./graphql/mutations";

Amplify.configure(
  {
    API: {
      GraphQL: {
        endpoint: env.AMPLIFY_DATA_GRAPHQL_ENDPOINT,
        region: env.AWS_REGION,
        defaultAuthMode: "iam",
      },
    },
  },
  {
    Auth: {
      credentialsProvider: {
        getCredentialsAndIdentityId: async () => ({
          credentials: {
            accessKeyId: env.AWS_ACCESS_KEY_ID,
            secretAccessKey: env.AWS_SECRET_ACCESS_KEY,
            sessionToken: env.AWS_SESSION_TOKEN,
          },
        }),
        clearCredentialsAndIdentityId: () => {
          /* noop */
        },
      },
    },
  }
);

const client = generateClient<Schema>({
  authMode: "iam",
});

export const handler: PostConfirmationTriggerHandler = async (event) => {
  await client.graphql({
    query: createUserProfile,
    variables: {
      input: {
        email: event.request.userAttributes.email,
        profileOwner: `${event.request.userAttributes.sub}::${event.userName}`,
      },
    },
  });

  return event;
};

The full diff from the related docs update can be found here.

[jserrano-alq]

Thank you so much @stocaaro ! I confirm this work around works just fine.

[spyderbr6]

i may have just needed to trigger a rebuild on my sandbox because (after the node packages were updated to this latest patch) vscode kept saying that error and so i just put logs in place in the handler before and after the retrieval to see what was there and miraculously it started working. thanks guys for the patch.

[spyderbr6]

there is something wonky with sandbox i think. i can't put my finger on it yet. at this point i have a stage branch that properly generates all the keys in .amplify/generated/env . i've branched from it and added a new function and now they all fail because amplify isnt generating any keys for anything in the new branch.

[spyderbr6]

i've been able to consistently reproduce this. while sandbox is running, i add an empty folder. it attempts to rebuild and starts failing everything.
amplifyfailonfunction.txt
here^

removing the folder yields an error but it appears to regenerate the files.

Error
Affected versions: bootstrap: <21
More information at: aws/aws-cdk#31885

If you don’t want to see a notice anymore, use "cdk acknowledge ". For example, "cdk acknowledge 31885".
Lambda bundled into an empty zip
Caused By: ❌ amplify-amplifyvitereacttemplate-Desktop-sandbox-acfc7abd3b failed: InvalidParameterValueException: Uploaded file must be a non-empty zip

Resolution: Try removing '.amplify/artifacts' then running the command again. If it still doesn't work, see aws/aws-cdk#18459 for more methods.

[Sandbox] Watching for file changes...

[spyderbr6]

i'm done mucking with it for the night. basically if anything is wrong with any function anywhere it kills all the generated environment files and nothing will work. basically replaces them with placeholders. makes it hard to troubleshoot because you cant get to a real error to see whats wrong with the function.

[spyderbr6]

ok. i'm closing out my engagement on this one. i think i'm in a good spot.

1. turn off sandbox.
2. manually reinstalled all the packages. npm was out of sync where it thought i had a package but i didnt. probably due to branch switching. outdated and such commands didnt matter, manually reinstall all the aws packages.
3. wipe out the cdk artifacts. the whole folder including the env folder.
4. ensure the function is completely error-free. i didnt negative test this after the first 3 steps so idk if it will still perform the wonkiness at this point. seems like the backend build process maybe could always generate environments first then attempt to run, which could enable better error checking rather than failing on step 1 when step 8 was really the problem.
5. goodluck if anyone else runs into this.

[stocaaro]

@spyderbr6 ,

Glad you have a process that is working and I really appreciate your notes. If I'm reading correctly, the steps you're following are roughly in line with the "Resolution:" guidance from the sandbox error output you posted above. Do you have any recommendations on how we might improve upon this guidance?

A couple questions for anyone encountering this,

• Are there any cases where you are seeing the type issue that don't correspond to an error captured in the sandbox output with a "Resolution:" suggestion?
• Would a change to the type error that you get when the env variables are missing be more clear if it specifically directed you to review the sandbox output for function failures?