Plugin request: Snowflake

[timgl]

I want to dump all my events and person information in Snowflake.

I've actually had a look at this. it seems the only way to do this is to create .csv's of events, upload them to s3 and then make a call to snowflake to ingest that. I'm trying to get a direct contact there to ask if there's an easier way, and to get added to their PartnerConnect program.

I think the easiest way to do this is to use their node connector, then do something like this to insert it one at a time.

[mariusandra]

Depends on #12 then

[jamesefhawkins]

I've had no response from the contact I got referred to, I've a third person I've been told I can get connected to through an investor (support = world networking championships, clearly), have asked him to intro! I've sent this question directly.

[mariusandra]

This is built: https://github.com/PostHog/snowflake-export-plugin

Some points:

• The following needs to be merged for it to work: Snowflake and crypto plugin-server#291
• Snowflake supports a few different methods of authentication. Currently only "password" is implemented. Adding the "private key" option shouldn't be that much extra work.
• Finally, we have the same case as with the bigquery export: there's no retry support with this plugin. In case snowflake or your network is down at some point, the events are lost. Good enough for now, but we need to find a universal solution to it (see Kafka last mile batching & retrying processEvent plugin-server#273)

[timgl]

I think we can safely ignore $snapshot and $feature_flag_called events for now (and maybe even autocapture). Other than that great work!

[mariusandra]

I can add some option in the plugin config to select what types of events to send. Who knows, someone might actually want to send their $snapshots somewhere for archival. The BigQuery plugin theoretically suffers from the same situation as it also just sends everything.

[mariusandra]

I added this (default values in screenshot):

[timgl]

yeah i think that's fine for now. would be nice to standardise that somehow but not urgent

[Twixes]

Snowflake contact woes aside, I would say this can be considered done #18