r/snowflake May 06 '26

r/snowflake needs your help: Where should this community go next?

155 Upvotes

Hi everyone. I'm Felipe — u/fhoffa.

I've been the top mod of r/snowflake since 2020. I began moderating this sub shortly after I joined Snowflake that same year. I left the company in 2024, but since then, the mod team has remained almost entirely composed of Snowflake employees.

That setup has worked. Snowflake employees moderating r/snowflake is not a problem. I was an employee while moderating this sub, and I currently moderate r/googlecloud, r/bigquery, and r/dataengineering despite having left Google in 2020. I believe it is possible to navigate conflicts of interest by putting the community first.

The problem starts when people with mod tools are also involved in coordinated campaigns to inorganically drive behavior in the same subreddit. That is where I believe we are now.

I have removed Snowflake employees from the mod team. I want to explain why, what happened, and how we move forward.

My Goals

  1. Protect the community from moderator-organized, incentivized, inorganic activity.
  2. Protect Snowflake employees from their own management retaliation - if they choose to say "no" and put community first
  3. Hand day-to-day moderation to active, independent community members.

What happened

u/aamoscodes founded this community. He made me a mod reluctantly at first — he didn't know if he could trust me. Over time I proved my priorities: community first. One of his concerns was that Snowflake might one day take over the sub and run it for corporate interests instead of the community's.

Recently, I saw facts that made that concern feel no longer hypothetical.

On March 27, 2026, a Snowflake employee mod removed u/bluepinkblack (Greg) from the team. Greg had seven years of experience at Reddit working on community programs before Snowflake hired him to manage their Reddit and forum community presence. He was arguably the most qualified person on the mod team to understand Reddit, community trust, and the risks of company-mandated participation.

I do not know the internal reason Greg was removed, but the sequence matters for this community: the most Reddit-experienced moderator was removed, and nineteen days later, a new Snowflake employee was added as a mod — the same person who later organized an incentivized campaign that explicitly included activity in this subreddit.

I also know Snowflake has fired employees in DevRel/community roles before. That makes it unfair to ask current Snowflake employees to hold mod tools in a community where their employer may have mandates that conflict with community-driven goals.

The "Build with CoCo Takeover"

Recently, u/ivannaatsnowflake sent a message to the "Snowflake Squad" (Snowflake's brand ambassador program) organizing a "Build with CoCo Takeover" that explicitly included r/snowflake.

The brief asked members to post 2–3 times a week, "correct misconceptions," and "spot misinformation in the wild." The incentives were explicit:

  • Featured spots on official Snowflake social channels.
  • A "CoCo Builder" badge.
  • Activity counting toward "Data Superhero" status.

When community member u/medvest posted about the campaign, another member tagged me directly: *"*u/fhoffa we should probably automod remove snowflake's posts." That was the alarm bell.

Ivanna replied in that thread:

"Our goal is to connect developers who are already building with Cortex Code with the conversations happening here. Real use cases and honest feedback from the community."

That sounds reasonable in isolation. But the actual brief describes something different: a posting quota, material rewards, and explicit direction to counter criticism in the subreddit moderated by the same person organizing the campaign.

Reddit's Moderator Code of Conduct is explicit about this. It states that "users expect that content in communities is authentic, and trust that moderators make choices about content based on community and sitewide rules." It lists conflicts of interest moderators must not act under, including "considerations and/or favors (e.g., special mentions from a company, promises of incentivized treatment)."

In my view, the brief creates the kind of incentive structure Reddit's rule is meant to prevent — special mentions on Snowflake's official channels, badges, and program advancement — to drive activity on the sub the organizing moderator moderates.

The Result: Inorganic Activity

The campaign appears to have already affected the subreddit.

The day before ODSC East 2026 began, Ivanna posted a thread titled "who is at ODSC East? share your thoughts." On day 2 of the conference, eight comments arrived in a six-hour window. Despite the conference having hundreds of sessions, these comments focused almost exclusively on one Snowflake product — CoCo — the same product named in the Squad brief:

Just met Coco at the Snowflake booth, really impressive...

So excited to hear about the newest CoCo features! It has quickly become my go-to AI tool!

One of those comments ends with a stray closing smart quote — the kind of artifact that appears when text is pasted in from somewhere else — and a hashtag, which is not a Reddit convention.

What I've Done

I have removed most mod powers from:

This is not a punishment. I am not saying every removed mod participated in this campaign, approved it, or acted in bad faith.

Snowflake employees should not be put in a position where their job is at risk. Think about what they're being asked to do right now: remove the spam that one of their own teammates is being paid by the same company to produce. That is an impossible position. Removing the mod role protects them from management retaliation.

Every one of those former mods is welcome to stay here as a member. They can post, comment, answer questions, explain Snowflake features, and represent the company openly. That participation is valuable.

To the community: please do not be mean to these individuals. Choosing between community values and a paycheck is an incredibly difficult position to be in. The Snowflake employees on the mod team have been incredibly helpful to this community for years, particularly in the tireless work of removing spam. I am trying to fix the pressure on them, not judge their character. Criticize the structure, the incentives, or my decision. Do not harass individual employees.

A Note on Integrity

I believe in people taking actions above their own short-term interests. When I was a Snowflake employee, I was called out for the conflict of being both an employee and a mod — not only at r/snowflake, but also at r/googlecloud, r/bigquery, and r/dataengineering. I pointed people to my mod logs to prove I never took a moderation action hostile to any of those communities. Other mods looked at the receipts and kept me. Not because conflicts don't exist, but because the question that matters is whether the person actually puts community above short-term company interests.

That is the line. People with conflicts can sit on the right side of it for years if they choose to. Employee participation is not the problem. Employee moderation during a company-sponsored, incentivized campaign aimed at the same subreddit is the problem.

This is not unique to Snowflake. Mods of r/bigquery, r/googlecloud, and r/snowflake have always had to navigate this tension — management teams that want to use Reddit for short-term goals. Every vendor subreddit faces it eventually. The solution is not banning employees from participating. The solution is having mods who are capable of putting community first and explaining to their management why spam is wrong. When that pushback stops working — or when the people doing the pushing back get removed and replaced — the structure has failed, and that is what happened here.

I believe the ideal mod is a company employee who genuinely cares about the community and is capable of saying no to misguided management. That kind of person exists — Greg was one of them. But if the model is instead going to be a paid community manager running incentivized campaigns, then the bare minimum is complying with FTC regulations for influencers — which require clear disclosure of material connections. The brief here doesn't include that guidance for participants. I am not trying to turn this into a legal argument; I am saying the disclosure and incentive structure matters for community trust.

Companies shouldn't be scared of ex-employees holding keys to a Reddit community. They should be scared of their own short-term goals destroying years of authentic community building.

What happens next?

I acted unilaterally because I didn't want anyone inside Snowflake to face consequences for being seen as "helping" me. This is entirely my call. The responsibility is mine alone.

There is no personal upside for me in doing this. Some Snowflake employees may be annoyed, and I understand that. But taking responsibility myself also means no current Snowflake employee has to choose between their employer's interests and the community's trust. If people are upset about this decision, they can blame me. That is the point of me acting alone.

I'm not putting this to a vote yet — Reddit polls can be brigaded, and given what's been documented above, that risk is not theoretical. Instead, I want to hear from you in the comments. Some paths forward:

  1. Independent Guard: I stay as temporary top mod and recruit new, independent mods from the community. No Snowflake employees in mod roles while these campaigns are active.
  2. Full Handover: I recruit independent mods and then step down entirely, leaving the sub fully community-run.
  3. Restore the previous mod team: The removed mods are reinstated and I step back.

There may be other options I haven't thought of. Say so.

Snowflake employees are welcome to comment too. If you have context I don't, share it. If you disagree with my read of what happened, say that. If you think I made the wrong call, make the case. I'd rather have the disagreement here in public than resolve it in modmail.

Help with the cleanup

In the meantime, I'll be moderating solo. Without the help of the Snowflake staff who usually handle the queue, it will be harder to stay on top of spam. Please use the standard Reddit "Report" button on any spam or rule-breaking content. This ensures it goes directly into my mod queue so I can review it quickly. Your help in flagging issues will be vital during this transition.

Everything I've done here is reversible. If the community concludes I'm wrong, I'll restore the mod team and step back. The reason I acted first and asked second is simple: I wanted this conversation to happen without any moderator being pressured by management to delete it. Once the discussion is underway, it's the community's call.

I don't want to spend too much time on this. I'll let the community reach consensus in the comments, and I'll delegate mod powers as soon as possible. This community deserves moderation that the community can trust. At the bare minimum: compliant with FTC guidelines and the Reddit Moderator Code of Conduct.

PS: Rule 3 of this subreddit says: "No Vendor Astroturfing — Intentionally hiding the sponsor of a marketing message by simulating community engagement (posts, comments, etc.) can result in content deletion and/or ban." Let's comply with that.

— Felipe


r/snowflake 14h ago

dbt 2.0 is out in alpha. What it means if you're running dbt on Snowflake (Core v2 vs Fusion, migration gotchas, dbt State pricing)

11 Upvotes

Disclosure: I'm from Datacoves, we run managed dbt Core, so we have a horse in this race. Wrote this up because the license change is confusing and we had to untangle it for our own customers.

The short version:

One engine now. dbt Core v2 and Fusion share the same Rust runtime. Core v2 is Apache 2.0, fully open source. Fusion is the enhanced proprietary distribution on top of it.

Fusion is the recommended default, and it's a funnel. Free to install, and the new license lets anyone offer it as a managed service, as long as you don't block the path to a dbt Labs account. Free login unlocks advanced features, paid platform account unlocks premium ones. Reasonable model, but go in with eyes open.

Metered pricing is inside the dbt Core now. dbt State (skip models when logic and data haven't changed) bills $0.094 per model reused per day. Orchestra open-sourced a free alternative (Sao Paolo, Apache 2.0).

Migration gotchas: every deprecation warning must be resolved before v2 runs your project. Python models don't work in Fusion yet. Community adapters are constrained (drivers need to be signed by dbt Labs). --models and -m error out, --partial-parse is gone.

The afternoon test: upgrade to v1.12, clear deprecations, run dbt parse --use-v2-parser. Tells you where you stand without committing to anything.

Full writeup with a Core v2 vs Fusion comparison and the four ways to handle state: https://datacoves.com/post/dbt-fusion

Has anyone run the v2 parser against a large project yet? And is anyone planning to use dbt State or trying out Sao Paolo?


r/snowflake 9h ago

Snowflake cost optimization always comes back to auto-suspend and oversized warehouses for me

4 Upvotes

Every time snowflake cost optimization lands on me it comes down to the same couple of boring things. Credits were up a good bit this quarter, finance flagged it and it wasn't more queries or more users. Usage barely moved.

Went through query history and it was mostly one warehouse that a BI tool kept awake almost all the time because auto-suspend was set to 3600 seconds instead of 60. Whoever set that has left. Two other warehouses were sized a couple of steps bigger than the work ever needed.

Dropped the suspend and resized the two big ones, that part was easy. What I keep going back and forth on is how far down you can size a warehouse before it starts hurting, because too small and the queries just queue and run longer and you give the savings straight back.

Where do you draw the line on warehouse size and are you catching the auto-suspend stuff as it happens or only when the bill jumps?


r/snowflake 8h ago

Best practice for Cortex Agent token and time budget?

1 Upvotes

Hello :)

I am currently setting up a Cortex Agent with the aim of using the Cortex Analyst to address complicated text-to-SQL/business questions.

At this point, I would like to establish common limitations of the orchestration budget with regards to the following factors:

– Tokens;

– Seconds.

The Snowflake solution has recommended a time budget of 5 minutes, which was unexpected.

At the moment, I am estimating something of around:

Budget: tokens: 16000, seconds: 300

The objective is primarily to protect against any unusual long reasoning or looping that would lead to unnecessary costs while avoiding rejection of valid questions.

To those who work with Cortex Agents on a day-to-day basis:

What limitations do you normally apply when setting tokens and time?

Do you use fixed limitations or determine precise limitations on the basis of the actual usage, such as P95 + certain margin?

Also, what experience do you have regarding reliability of the mechanism when applying time limits below 5 minutes?

Thanks for the feedback !


r/snowflake 15h ago

Delete types in Apache Iceberg v3

Post image
3 Upvotes

r/snowflake 1d ago

Testing Out Snowflake Openflow Connector For Free

2 Upvotes

Hi all, i am testing out snowflake as a personal development project. I wanted to see how cdc works from a cloud database like azure postgresql. I tried to use openflow but I think it requires a license that free trial doesnt allow. I tried the legacy method of postgresql connector but I think its too complicated to setup (as a cloud newbie). Do you guys have any recommendations for testing out openflow? I dont really want to buy a license because it might cost me a lot bcs I dont really know what im doing here.


r/snowflake 1d ago

Streamlit as a way of standardizing Data Registry for Lab Data?

3 Upvotes

Currently I use Excel files and manually capture data in these Excel files. I have other peers who also do a similar procedure and we all have different formats. Is a streamlit app a good solution to standardize the excel format ? Is it even feasible as something that can be deployed to multiple users through tablets so that they can register the data via a streamlit app ?


r/snowflake 2d ago

Am I understanding the view Snowflake.account_usage.query_history correctly?

7 Upvotes

I think one of my warehouses is overutilized, so I was looking at the view snowflake.account_usage.query_history to see relevant stats during queries. I know the view is documented here in the docs, but I still have questions. Is the field query_load_percent reporting how much resources were used for that specific query in the row? Or is it reporting the resources that were being used across all fields at the time the query was being executed?

Are there any other fields in that table or other views that I should look at to see if my warehouse is being used efficiently and sized correctly?


r/snowflake 2d ago

Sharepoint excel to snowflake automation

8 Upvotes

I have many excel files in sharepoint which keep updating regularly i want to connect them with snowflake tables so that snowflake tables have same data as those excel files ,suggest how to automate this

Update:
I used a JSON based configure file stored in share point which has details for each excel file like
Target table :
Columns:
KeyColumns:
UpdateColumns:
Source:{siteurl: ,folderpath: ,filename: ,sheetName:}

I parsed this sort of json in power automate then passed these variables to a single office script

Which runs for each excel file and creates a sql statement and that sql gets executed in power automate

So now if i need ro fetch data from any new excel i just need to enter its details into json configure file

This method is working good for me tried on many excels stored in different sharepoints


r/snowflake 2d ago

Uploading 250MB+ files to Snowflake Internal Stages using only Snowsight

7 Upvotes

Traditionally, when I needed to upload large files into Snowflake, I relied on SnowSQL, Snowflake CLI, or an external stage such as S3.

Recently, I experimented with a different approach using Streamlit in Snowflake, st.file_uploader, and the Container Runtime. This makes it possible to upload multiple files—including files larger than the standard Snowsight limits—directly into an internal stage without leaving the Snowflake ecosystem.

I wrote up the architecture, limitations, and implementation details here:

👉 Snowflake: Uploading 250MB+ files to Internal Stages using only Snowsight

I'm curious how others handle large file uploads today:

  • Snowflake CLI / SnowSQL
  • External stages (S3, Azure Blob, GCS)
  • Snowpark or Native Apps
  • Streamlit in Snowflake

Would love to hear about other approaches and trade-offs.

#snowflake #streamlit #dataengineering


r/snowflake 2d ago

What data products or Native Apps are you actually willing to pay a premium retainer for?

4 Upvotes

Hey everyone, I’ve been deep in the Snowflake ecosystem for a while, primarily leveraging snowpark, streamlit, and cortex AI for internal workflows. I’m now looking to transition into building and launching a commercial product directly onto the Snowflake Marketplace.

Instead of guessing what the market needs or launching another generic LLM prompt-wrapper, I want to talk directly to the enterprise architects, data product managers, and founders here. Given how frictionless the native app framework is for bypassing grueling 6 month enterprise security reviews (since code executes within the consumer’s perimeter), I’m looking to solve a real, high-value data friction point.

Appreciate any blunt data engineering realities or marketplace wishlists you can throw my way!


r/snowflake 2d ago

Kinda confused in this badges for learning purpose....

3 Upvotes

While doing the Snowflake badges, I feel like I'm just copying and pasting code. I'm not going to lie—I have learned the overall UI and gained a basic understanding of how Snowflake works, but not much beyond that.

Should I continue, or not?

It also takes a lot of time just to learn the basics, and I feel like I'm wasting a lot of time doing it.


r/snowflake 2d ago

Job Opportunity - Developer/ Lead / Architect

Post image
5 Upvotes

r/snowflake 2d ago

Explained Snowflake Iceberg Tables Using a Stranger Things Theme 👀

Post image
0 Upvotes

I wanted to make Iceberg Tables easier to understand, so I created this Stranger Things-inspired infographic. The idea is simple: what you query is just the tip of the iceberg, while snapshots, metadata, schema evolution, and Parquet files are hidden underneath. Feedback is welcome—especially if I've oversimplified or missed anything


r/snowflake 3d ago

Automating Snowflake account provisioning and role assignment with Entra ID, SCIM, Microsoft Forms, Power Automate, and Teams

5 Upvotes

Hi r/snowflake,

I wrote a technical article about automating Snowflake account provisioning, authentication, and authorization by combining:

- Snowflake

- Microsoft Entra ID

- SSO / SAML

- SCIM provisioning

- Microsoft Forms

- Power Automate

- Microsoft Teams

Article:

[Automating Snowflake authentication, authorization, and account provisioning with Forms + Power Automate + Teams + Entra ID](https://qiita.com/mshdtksk/items/de1e9c730e2aa7c2af78)

The main idea is to use Entra ID groups as the source of authorization, where each group corresponds to a Snowflake role through SCIM provisioning.

The flow looks like this:

  1. A user submits a request through Microsoft Forms
  2. Power Automate receives the request
  3. The user is added to a Microsoft Teams team / Entra ID group
  4. SCIM sync provisions the user and group into Snowflake
  5. The corresponding Snowflake role is assigned automatically
  6. The user logs in to Snowflake through Entra ID SSO

This approach helped me reduce manual account creation and role assignment work in my multi-cloud lab environment.

A few key takeaways:

- SSO handles authentication

- SCIM provisions users and groups

- Entra ID groups can be mapped to Snowflake roles

- Teams membership can become an operational interface for role assignment

- Power Automate can connect the request and approval process

- Snowflake-side GRANTs still need to be designed carefully

- SCIM sync is not always immediate, so some delay should be expected

This is not meant to be a perfect production-ready pattern, but rather a practical example of how existing Microsoft 365 / Entra ID assets can be used to simplify Snowflake access management.

I would be interested to hear how others are handling Snowflake user provisioning and role management.

Are you using SCIM with Entra ID, Okta, or another IdP?

Do you manage Snowflake roles directly, or do you map them from external groups?


r/snowflake 5d ago

I cleared my Snowflake Certification COF-C03 with 17 days of prep (840+)

27 Upvotes

I'm just grateful that I cleared it. Graduated recently and in an extremely panicky stage right now wrt jobs. Someone suggested, I should get this certificate. Most of my experience is in software development and ML and the only previous experience I had working with something related to databases was with IBM Filenet.

anyways here's the study guide I followed and some tips if helpful

- Started with Tom Bailey's videos and hands on

It is considerably easy and might give you false confidence for the actual exam

- Did a few practice tests online, many free tests and some paid ones have very outdated questions, be aware

- While practice is good, some tests are too easy while some are unnecessarily complicated

- purchased a practice test series from udemy which was wayyy outdated but they refused to give me my money back🫠

- Purchased Tom's practice set, while it gives you a vast array of info, some questions might be outdated like the maximum size of a variant row which says 16mb in the test but it's been resized to 128mb

- I was panicking about numbers, I don't think these are important except for a few like time travel or partition size. Memorising every function's time period is kind of overwhelming and unnecessary, me feels

I didn't get even one question about these

- I then tried giving practice tests with chatgpt but even with context or searching it will only give you very obvious questions and options that you can easily eliminate. The actual exam holds all the power in the choices

- I also used chatgpt sol as my final prep stage which was kinda better but exam questions are not as straightforward as exam sets or LLM generated questions are.

I realised that if I practice more tests I'll only end up panicking as there's never-ending information in snowflake so I just scheduled it.

Exam scenario:

Initial questions put me in a state of panic, the video courses and the exam sets didn't prepare me for SCD, DMF, Telemetry, Usage reports (not what you are thinking) etc but from what I noticed from plenty Redditor's posts, snowflake typically does give harder questions at the top. The second half was too straightforward and simple but I kind of panicked at the start and lost a lot of time.

Majority of the questions were related to

- Materialized vs SOS vs QAS vs Warehouse (they try to trick you but mostly the answer is materialized😭)

- result vs query vs metadata caches

- external tables and staging

- sharing and listing (I thought it was only a small part of the syllabus)

- dynamic table was overrepresented

- they are apparently huge fans of Apache iceberg and external staging

- some streaming, privileges and network policy questions but they try to trick you with options

- some easy questions about masking and row policies

- tricky part is roles, priveleges and heirarchy

- Cortex was barely there

I'm honestly surprised about the latter half of the exam, it was absolutely undeniably, a piece of cake. If I hadn't panicked in the first half, I would probably have a late 800 or early 900 score.

Prep time

I took around 10+ days to finish the videos between my depressive days and long unbearable working hours.

Then practiced test sets for 4 days, scheduled the exam the same day, panicked and rescheduled it to the next day but it was actually a good decision, if I had delayed nothing good was going to come out of it. No test series had any questions that I failed the answers for.

I don't know what the future holds and I know I could've done better but I'm just happy right now.

Edit: If the passing criteria was 50%, you would probably clear it by only studying about how queries work


r/snowflake 4d ago

AuthenticationPolicy

0 Upvotes

With Snowflake, you don’t play around with security: a AuthenticationPolicy lets you control who gets in, how, with what, from where, and under which conditions — all at once.

With a single policy, you define:

  • Who can log in — allowed email domains, external accounts blocked
  • How they authenticate — SSO for humans, KEYPAIR for services, passwords forbidden

r/snowflake 5d ago

Free course: Production-grade data ingestion using OSS dlt and Snowflake.

Thumbnail
dlthub.learnworlds.com
15 Upvotes

We built this free course because these questions come up constantly from Snowflake users: how to stage properly, when to use SPCS, what Native Apps actually change for ingestion.

Most Snowflake ingestion setups run outside Snowflake — an Airflow box, a Lambda, a SaaS tool, and pipe data in. Snowpark Container Services changed that: you can run the ingestion itself inside your Snowflake perimeter, next to the data, on compute you already govern.

The course covers production-grade ingestion patterns for Snowflake using dlt (the open-source Python library), in this progression:

  • Core loading patterns — merge/append/replace, schema evolution, incremental state
  • External S3 staging — when volumes make direct loading the wrong tool
  • In-Snowflake execution with SPCS — running your pipelines inside your own account
  • Snowflake Native Apps — packaging ingestion as an integrated workflow

Python-based and hands-on. If you've been wiring dlt + Snowflake together from docs and guessing at the staging or SPCS setup, this consolidates it into one path.

📖 Course link

We also offer a few other free courses that aim to teach data ingestion and related best practices more broadly

  • Agentic Data Engineering with dltHub — building, validating, and deploying pipelines with AI agents: from one prompt to a running system you can trust
  • dlt Fundamentals — start here if dlt is new to you: core Pythonic loading concepts before you point anything at Snowflake
  • dlt Advanced — custom sources, complex APIs and databases, and controlling exactly how data lands in your warehouse
  • Deploy dlt pipelines — if you run ingestion outside Snowflake, this covers deploying on the common orchestrators
  • Freecodecamp: Data Engineering with Python and AI — the broadest one: Python data engineering from ingestion basics through transformation, deployment, and automation

disclaimer: I work at dltHub, we're a Snowflake partner


r/snowflake 5d ago

Need ideas/Labs for cortex AI

6 Upvotes

I got access to Snowflake Intelligence, coco and cortex analyst and other AI stuff in snowflake I want to become an expert in this part, I need ideas or any realtime scenario labs... Thanks in advance...


r/snowflake 6d ago

🚀 What do most Snowflake courses fail to teach?

8 Upvotes

I'm thinking about creating a completely free Snowflake course and uploading everything for the community.

Before I start recording, I wanted to ask:

What do you actually want to learn?

Should I focus on:

- Snowflake from scratch

- Advanced SQL

- Snowpipe & S3

- Streams & Tasks

- Dynamic Tables

- Performance tuning

- Security (RBAC)

- Airflow integration

- Real-world data engineering projects

- Interview questions & mock interviews

Also, what other technologies should I include alongside Snowflake? (AWS, Azure, GCP, Python, dbt, Airflow, Apache NiFi, etc.)

The goal is to build something that's practical, industry-focused, and 100% free.

I'd really appreciate your suggestions. What do you think most Snowflake courses are missing?


r/snowflake 7d ago

Alerting for application query

8 Upvotes

Hi,

Few teammates suggesting to set alerting based on the historical Avg. of the application query run time. Want to know from experts, is this something we should really do and will that add real value?

Say for e.g. if the normal AVG respnse time is ~10 minutes and todays its has exceeded ~30minutes, then it means something wrong is there (say it may be a bad plan etc.), so that should get alerted.

And for above to get implemented, I am thinking to refer column query_parameterized_hash and first store the last one month of query_history data from account_usage view as reference and get that refreshed daily once(say the table query_baselines). say mainly query_parameterized_hash and avg response time avg_seconds, p95_seconds etc. And have another task created , which will fetch data from information_schema.query_history and put it in a temp table and then join this data with the query_baselines table to see if any query ran > (avg_seconds * 3) , will throw an alert. This task may be executed once in ~15 minutes or so. (I understand it wont cater adhoc or newly added queries as hash will change).

Is above approach is good and really worth or any other easy and less costly way exists to achieve such alerting?


r/snowflake 7d ago

Help pip installing Python module in Snowflake Notebooks

6 Upvotes

When I attempt to run a machine learning model, XGBoost, in Python using the XGBClassifier and GridSearchCV functions I receive the following error message:

The version of package 'xgboost' in the local environment is 3.1.3, which does not fit the criteria for the requirement 'xgboost==3.1.2'. Your UDF might not work when the package version is different between the server and your local environment.

When I attempt to pip install xgboost:

!pip install xgboost==3.1.2

I get the following message:

Looking in links: /opt/wheels
[33mWARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8134ea7e50>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/xgboost/[0m[33m
[0m[33mWARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8134ed8190>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/xgboost/[0m[33m
[0m[33mWARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8134ed8340>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/xgboost/[0m[33m
[0m[33mWARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8134ed84f0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/xgboost/[0m[33m
[0m[33mWARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8134ed86a0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/xgboost/[0m[33m
[0m[31mERROR: Could not find a version that satisfies the requirement xgboost==3.1.2 (from versions: none)[0m[31m
[0m[31mERROR: No matching distribution found for xgboost==3.1.2[0m[31m
[0m

What's going on?


r/snowflake 7d ago

Snowflake Snowpro Core Practice Exam, 1 Question that I have no clue on...

4 Upvotes

Hey there!

Got this question on a COF-CO3 Snowpro Core Practice Exam, and I truly have no clue what the answer is (google and ChatGPT don't know either, they disagree with the question).

Which table characteristic will prevent the query acceleration service in Snowflake from accelerating queries?

A.) The table contains a large number of micro-partitions.

B.) The table has the search optimization service configured.

C.) A clustering key is defined on a column with the GEOMETRY data type.

D.) The table has been created using the CREATE TABLE ... AS SELECT command.

Any help would be greatly appreciated!


r/snowflake 7d ago

reduce idle compute by suspending warehouses

5 Upvotes

Hi friends, I'm a co-founder at Greybeam. A few weeks ago we released an open source Snowflake cost observability tool with surprisingly good reception (you know how harsh reddit can be!). We got a lot of feedback on whether we could build a way to reduce idle compute as it's especially relevant for multi-cluster users. Snowflake will aggressively spin up clusters and can often take over 10 minutes to wind them down despite no activity because unfortunately the only levers are `SCALING_POLICY = STANDARD or ECONOMY`.

So today we launched this exact feature and it's free to use either at Greysight or self-hosted. the tldr is we poll Snowflake and if the following criteria below are met then we issue a SUSPEND on the warehouse.

  • status is STARTED
  • running queries = 0
  • queued queries = 0
  • resumed on >= 60s ago
  • and a few others

Would love for you to try it and share any feedback, the feature is still early and a bit bare bones--really just a enable or disable config but we intend on adding more. Next up among other observability features is finer controls on scaling.

More details here: https://www.greybeam.ai/blog/snowflake-auto-savings

Github: https://github.com/greybeam/greysight


r/snowflake 7d ago

GxP validation approach for Snowflake, dbt & custom sql

12 Upvotes

For those who have implemented Snowflake and dbt in a regulated environment:

  • Did you use one overall change control or separate change controls for the platform, dbt transformations, and stored procedures?
  • How did you classify Snowflake, dbt models, macros, stored procedures, and custom SQL under GAMP 5?
  • Did you treat the platform as Category 4 and custom-developed code as Category 5?
  • What were the core documents in your validation package?
  • How do you manage frequent dbt changes without revalidating the full platform?
  • Do you validate the platform once and then validate individual data products through risk-based change controls?

I would appreciate practical lessons learned from anyone who has gone through the validation qualification.