Snowflake Quickstart

Use this page when the Snowflake objects for the first run do not exist yet. It creates the runtime role, service user, warehouse, control table, target table, and Snowpipe Streaming pipe.

Before You Start

You need an active Snowflake account and a setup role that can create users, roles, warehouses, databases, schemas, tables, pipes, and grants. ACCOUNTADMIN is acceptable for a one-time bootstrap when your organization does not provide a narrower setup role.

Run the local commands on this page from the EvSnow repo root after cloning the repo and running uv sync.

The commands below use the Snowflake CLI:

snow connection test --connection <setup-connection>

You can also run the same SQL in Snowflake Worksheets. Replace <setup-connection> with the connection name for your setup role.

Generate The RSA Key

For an interactive local setup, run:

./generate_snowflake_keys.sh

The script creates:

For headless validation or CI, provide the password through EVSNOW_KEY_PASSWORD:

EVSNOW_KEY_PASSWORD="replace-with-key-password" ./generate_snowflake_keys.sh

Run The Snowflake Bootstrap

Render setup_snowflake.sql with the generated public key, then run it with the setup connection:

The checked-in script starts with USE ROLE ACCOUNTADMIN. If your organization provides a narrower setup role, replace that line with USE ROLE <setup-role> in .quickstart/setup_snowflake.sql before running it.

mkdir -p .quickstart
PUBLIC_KEY_VALUE="$(tr -d '\n' < snowflake/rsa_key_pub_value.txt)"
sed "s|<PUBLIC_KEY_VALUE>|${PUBLIC_KEY_VALUE}|g" \
  setup_snowflake.sql > .quickstart/setup_snowflake.sql

snow sql \
  --connection <setup-connection> \
  --filename .quickstart/setup_snowflake.sql

The script creates missing objects and verifies them. It uses CREATE TABLE IF NOT EXISTS for CONTROL.PUBLIC.INGESTION_STATUS, so rerunning the script preserves existing checkpoint rows.

It also grants the STREAM role the privileges needed by the runtime, including control-table setup and validation.

Create The Target Table And Pipe

Run the Snowpipe Streaming setup script:

snow sql \
  --connection <setup-connection> \
  --filename setup_snowpipe_streaming.sql

The script creates the Snowflake-managed Iceberg target table and the high-performance Snowpipe Streaming pipe if they do not already exist:

The table uses Snowflake-managed Iceberg storage:

CREATE ICEBERG TABLE IF NOT EXISTS INGESTION.PUBLIC.EVENTS_TABLE1
  CATALOG = SNOWFLAKE
  EXTERNAL_VOLUME = SNOWFLAKE_MANAGED
  ICEBERG_VERSION = 3;

Use Complete Snowflake setup for the long-form object reference.

Verify Snowflake Objects

Run these checks with the setup connection:

snow sql --connection <setup-connection> --query "
DESC USER STREAMEV;
SHOW GRANTS TO ROLE STREAM;
DESC TABLE CONTROL.PUBLIC.INGESTION_STATUS;
DESC TABLE INGESTION.PUBLIC.EVENTS_TABLE1;
SHOW PIPES LIKE 'EVENTS_TABLE_PIPE' IN SCHEMA INGESTION.PUBLIC;
SHOW GRANTS ON PIPE INGESTION.PUBLIC.EVENTS_TABLE_PIPE;
"

The pipe check must return EVENTS_TABLE_PIPE, and the grant checks must show the runtime role has table access plus OPERATE and MONITOR on the pipe. DESC USER STREAMEV should show TYPE = SERVICE.

Keep this checklist as the known-good proof before moving on:

Configure EvSnow

cp config/evsnow.example.toml config/evsnow.toml

Set the Snowflake target in config/evsnow.toml:

[snowflake_configs.SNOWFLAKE_1]
database = "INGESTION"
schema_name = "PUBLIC"
table_name = "EVENTS_TABLE1"
batch_size = 100

Create .env with only secrets and local credentials:

SNOWFLAKE_ACCOUNT=<account_locator>
SNOWFLAKE_USER=STREAMEV
SNOWFLAKE_PRIVATE_KEY_FILE=snowflake/rsa_key_encrypted.p8
SNOWFLAKE_PRIVATE_KEY_PASSWORD=<key-password>
SNOWFLAKE_WAREHOUSE=COMPUTE_WH
SNOWFLAKE_ROLE=STREAM
SNOWFLAKE_PIPE_NAME=EVENTS_TABLE_PIPE

For this one-target quickstart, EvSnow derives the Snowflake session database/schema from config/evsnow.toml. Set SNOWFLAKE_DATABASE and SNOWFLAKE_SCHEMA_NAME only when you map to multiple database/schema pairs or need an explicit session context.

The full environment template remains in .env.example.

Verify EvSnow Configuration

uv run evsnow validate-config --config-file config/evsnow.toml --env-file .env

This validates the resolved EvSnow configuration and control-table access. Keep the Snowflake object checks above as the proof that the Iceberg table, pipe, and pipe grants exist.

After validation passes without warnings, continue with First run.

Maintainer Harness

When setup SQL or Snowflake setup docs change, maintainers can run the same path in a scratch copy and keep the command log:

uv run python tools/quickstart_harness.py --connection <setup-connection>

The harness writes summary.json and commands.jsonl under .quickstart-runs/. A passing run reports "status": "passed" and includes Snowflake control table verified/created successfully in the validation output. It does not create Event Hubs or prove row arrival; use First run for that runtime proof.