How to Import Snowflake Python Libraries in AWS Lambda

The Snowflake Python connector will not import in AWS Lambda if you zip it up from a Mac or Windows machine. It contains compiled components that must match Amazon Linux, so the package has to be built in a matching environment. The reliable approach is a Lambda layer built inside the official AWS base image with Docker.

Why the obvious approach fails#

Unable to import module 'lambda_function': cannot import name '...' from
partially initialized module 'snowflake.connector'

Or something mentioning a shared object file. The Snowflake connector depends on packages such as pyarrow and cryptography that ship compiled extensions. Pip downloads the wheel matching your machine, so a zip built on macOS contains Darwin binaries that Amazon Linux cannot load.

Build the layer with Docker#

mkdir -p snowflake-layer/python
cd snowflake-layer

docker run --rm --platform linux/amd64 \
  -v "$PWD":/var/task \
  public.ecr.aws/lambda/python:3.12 \
  /bin/sh -c "pip install snowflake-connector-python -t /var/task/python --no-cache-dir"

Three details matter. The image is the official Lambda base, so the environment matches at run time. The platform flag is essential on an Apple Silicon machine, or you get ARM wheels. And the target folder must be named python — Lambda looks for that exact name inside a layer.

zip -r snowflake-layer.zip python

aws lambda publish-layer-version \
  --layer-name snowflake-connector \
  --zip-file fileb://snowflake-layer.zip \
  --compatible-runtimes python3.12 \
  --compatible-architectures x86_64

Attach the layer to your function, and make sure the function’s Python version and architecture match what you built.

Staying under the size limit#

An unpacked function plus its layers must fit in 250 MB. The full connector with pyarrow gets close, so trim what you do not need:

cd python
find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name "tests" -exec rm -rf {} +
find . -name "*.pyc" -delete
find . -type d -name "*.dist-info" -exec rm -rf {} +

If you do not need Arrow-format results, install without the pandas extra — that alone saves a great deal:

pip install snowflake-connector-python -t python --no-cache-dir
# rather than the [pandas] extra

Credentials#

Never put a Snowflake password in the function code or in a plain environment variable. Use Secrets Manager and read it at run time:

import json
import boto3


def get_secret(secret_id: str) -> dict:
    client = boto3.client("secretsmanager")
    response = client.get_secret_value(SecretId=secret_id)
    return json.loads(response["SecretString"])

Grant the function’s execution role permission to read that one secret, and nothing wider.

A working handler#

import json
import os
import boto3
import snowflake.connector

_connection = None      # survives between warm invocations


def get_connection():
    global _connection

    if _connection is not None and not _connection.is_closed():
        return _connection

    secret = json.loads(
        boto3.client("secretsmanager")
        .get_secret_value(SecretId=os.environ["SNOWFLAKE_SECRET"])["SecretString"]
    )

    _connection = snowflake.connector.connect(
        account=secret["account"],
        user=secret["user"],
        password=secret["password"],
        warehouse=secret["warehouse"],
        database=secret["database"],
        schema=secret.get("schema", "PUBLIC"),
        login_timeout=15,
        network_timeout=30,
    )
    return _connection


def handler(event, context):
    try:
        conn = get_connection()
        cursor = conn.cursor(snowflake.connector.DictCursor)
        cursor.execute(
            "SELECT id, name FROM products WHERE price < %s LIMIT 100",
            (event.get("max_price", 100),),
        )
        rows = cursor.fetchall()
        cursor.close()

        return {"statusCode": 200, "body": json.dumps(rows, default=str)}

    except snowflake.connector.errors.Error as error:
        print("Snowflake error:", error)
        return {"statusCode": 502, "body": json.dumps({"error": "query failed"})}

Two things worth copying. The connection is held in a module-level variable, so a warm invocation reuses it rather than paying the handshake cost again — which is significant with Snowflake. And the query uses placeholders rather than string formatting.

Timeouts and networking#

• Function timeout. The default of three seconds is not enough. Allow at least thirty for a cold start plus a connection.
• Memory. More memory means proportionally more CPU, so 512 MB often runs faster and cheaper than 128 MB for import-heavy functions.
• VPC. If the function is in a VPC it has no internet route by default. You need a NAT gateway or a PrivateLink endpoint, or every connection attempt will simply time out.
• Login timeout. Set it explicitly, as above, so a network problem fails fast instead of consuming the whole function timeout.

Test the layer before writing logic#

def handler(event, context):
    import snowflake.connector
    return {"version": snowflake.connector.__version__}

Deploy that first. If it returns a version number, the layer is correct and every later problem is in your code or your network configuration. Separating those two questions saves a great deal of time.

Where to go next#

• Python virtual environments — the local equivalent of a layer.
• Fixing ModuleNotFoundError — the ordinary version of this problem.
• Deployment and packaging — the general principles.