Postgres

🧭 Situation I’m setting a Metabase dashboard to view the status of my data stored in Supabase. Options for Secure Connection to Metabase Option 1. IP Allowlist via EC2 Security Group Allow inbound access only from your home or office IP. Effect: Only you can reach the login page. Tradeoff: You’ll need to update the rule if you travel or use a new network. Option 2. Put Metabase Behind Reverse Proxy + Basic Auth Run an Nginx reverse proxy on the same EC2 instance. ...

Here’s how I added search to my KBooks site. All my book data was already flowing into Supabase from the National Library of Korea API through a dbt pipeline, so the challenge was: How do I make this table searchable efficiently without adding another service at this stage? This is how I got database-level search ready, step by step, and why each decision made sense. 1. Start from a clean source: silver_books My raw data lands as JSON in raw_nl_books. From there, I built a silver_books table using dbt. Each row represents one unique book identified by its ISBN-13, and only valid ISBNs are kept. ...

I was building a data pipeline using Kafka and Spark structured streaming. Fully containerized. The stack: Kafka for streaming transaction data Spark Structured Streaming for real-time processing and fraud detection Postgres as the data warehouse Everything was smooth. Until one tiny villain showed up: UUID fields. Yes — UUIDs. Here’s exactly what happened (so you can avoid the same headache). ✅ The Original Design I designed the tables in Postgres like this: ...

Common issues you will often encounter when running Airflow with Docker. ❗ Issue 1 — .env file is not visible inside Airflow container 🔍 Symptom Summary The .env file exists at the project root. But inside the Airflow container, load_dotenv() fails to read it. The reason: Docker automatically passes .env as environment variables. But Docker does not copy or mount the file itself into the container. Therefore, load_dotenv() has no file to read. ✅ Solution 1️⃣ Add volume mount for .env in docker-compose.yml This way, the .env file becomes available inside the container at the correct path. ...