data-engineering-medallion-pipeline

Data Engineering Medallion Pipeline Skill Skill by ara.so — Data Skills collection. This skill enables AI agents to work with a complete data engineering pipeline implementing the Medallion Architecture (Bronze → Silver → Gold) using modern open-source tools: MinIO (S3-compatible storage), Airbyte (data ingestion), PostgreSQL (data warehouse), DBT (transformations), Apache Airflow (orchestration), and Grafana (monitoring). What This Project Does The data-engineering-medallion project provides a complete end-to-end data pipeline that: Ingests raw data from MinIO object storage into PostgreSQL using Airbyte Transforms data through three layers (Bronze/Silver/Gold) using DBT Orchestrates the entire pipeline with Apache Airflow DAGs Validates data quality with automated DBT tests Monitors infrastructure health with Prometheus and Grafana Visualizes business metrics in Power BI dashboards Show more Installs 551 Repository aradotso/data-skills GitHub Stars 1 First Seen May 23, 2026 Security Audits Gen Agent Trust Hub Pass Socket Pass Snyk Warn