How To Import Snowflake Python Libraries In AWS Lambda

AWS Lambda provides a serverless computing service, allowing you to run code without provisioning servers. It supports Python as one of its runtimes. To connect your Lambda functions to Snowflake for data warehousing, you need to import the Snowflake connector and SnowSQL libraries.

This poses a challenge because the Lambda environment contains only the Python standard library by default. So we need to package any other dependencies along with our deployment bundle.

In this guide, we’ll cover the necessary steps to import and bundle the Snowflake Python connector (Snowflake-Connector-Python) and SnowSQL (SnowSQL) libraries for use in Lambda:

Overview:

• Set up a Python virtual environment
• Install Snowflake connector and SnowSQL
• Bundle as deployment package
• Upload deployment package to Lambda
• Import libraries and connect to Snowflake

Prerequisites:

• AWS Lambda console access
• Basic Lambda & Python knowledge
• Snowflake account credentials

Set Up a Virtual Environment and Install Dependencies

First, create a dedicated directory for your Lambda project. We’ll use a Python 3.7 virtual environment to mimic the Lambda runtime environment locally.

Steps:

Create project folder

$ mkdir snowflakeLambda
$ cd snowflakeLambda

Create & activate Python 3.7 virtual env

$ python3.7 -m venv .venv
$ source .venv/bin/activate

Upgrade pip and setuptools

(.venv) $ pip install --upgrade pip setuptools

Install Snowflake connector (latest)

(.venv) $ pip install snowflake-connector-python

Install SnowSQL Python library

(.venv) $ pip install snowsql

This installs the Snowflake libraries we want and their dependencies into our local virtual environment.

Bundle Dependencies into Deployment Package

AWS Lambda requires all function dependencies to be bundled together in a deployment package zip file. We need to create a bundle containing our function code and the Snowflake libraries.

Steps:

Write basic Lambda function code (e.g. lambda_function.py)

import snowflake.connector

def lambda_handler(event, context):
  ...

• In the project directory, create a folder pythonCopy virtual environment site-packages into python folder

   (.venv) $ cp -R .venv/lib/python3.7/site-packages/* python  

• Zip the python folder to create a package

   $ zip -r9 deployment.zip python/*  

This bundles all our libraries so Lambda can access them at runtime. We only include the contents under site-packages, not the entire virtual environment.

Upload and Test Lambda Deployment Package

With our deployment bundle zipped, we can now upload it to Lambda and test importing the Snowflake modules.

Steps:

In Lambda console, create a new function

For runtime, select Python 3.7

Skip creating functions code from scratch

Under function package select “Upload a .zip file”

Upload the deployment.zip bundle we created

Save function

In function code editor, add test code:

import snowflake.connector
import snowsql

print(snowflake.connector.__version__) 
print(snowsql.__version__)

Run a test execution

Check logs to verify Snowflake connector versions print

This validates that Lambda now loads those libraries at runtime. We can begin coding our Snowflake integration!

Connecting to Snowflake from Lambda

With the foundations in place, we are ready to connect a Lambda function to our Snowflake account using the imported libraries.

Steps:

Import connector

import snowflake.connector

Define a Snowflake connection function

def get_conn():
    return snowflake.connector.connect(
        user='<your_user>',
        account='<your_account>',
        private_key='<private_key>', 
        role='<role>',
        warehouse='<warehouse>'
    )

Utilize connection to execute query

def run_query():
    conn = get_conn() 
    cs = conn.cursor()
    cs.execute("SELECT current_version()")  
    one = cs.fetchone() 
    print(one[0])

By bundling the libraries into our deployment package, we enabled Python code running on Lambda servers to interact directly with Snowflake!

There are many further possibilities for querying data, loading files, triggering notifications, and more by connecting Lambda serverless with Snowflake’s data warehouse services in this manner.

This cross-cloud platform integration grants efficient scaling, automation capabilities, and streamlined devops pipelines around your analytics stack.