Saisoku is a Python module that helps you build complex pipelines of batch file/directory transfer/sync jobs.

Tasks around metadata.

Pulling games from the Lichess API into a PostgreSQL database for data analysis.

amplicon analysisng pipelines based on luigi with qiime2-dada2/ qiime2-deblur/vsearch/usearch

Demo of using Luigi to build ETL Pipelines

Projekt zur Live-Darstellung von HVV Fahrzeugen auf einer Karte.

Shows how luigi can be used with kubernetes

EDA and Luigi pipeline for M5 Forecasting (Accuracy) Competition hosted on Kaggle.

Easy and powerful template for ML projects

🔍 Example of an ETL pipeline using Spotify's Luigi

Universal Luigi ETL pipeline. Validates data received from external sources. Extracts, transforms them and lands.

Sebagai seorang Data Engineer di Erdigma, tugas ini bertujuan untuk menggabungkan beberapa file CSV yang berasal dari Google Drive ke dalam satu file agar dapat diinput ke database. Pengolahan data ini dilakukan dengan menggunakan Python dan Luigi sebagai workflow automation.

Left for reference on how to write a pipeline in luigi

Pycon 2017- Creating ETL tasks with Luigi package

This repository is based on scraping data from a static website through Luigi. This was created to display my ability to utilize the Luigi pipeline to automatically collect data and other tasks.