Airflow Dagster pipeline deduping and enriching CRM leads
Course Content
0 / 100 completedCDC pipeline from PostgreSQL to BigQuery with late arrivals
Kafka → Delta Lake streaming with watermarking and compaction
Airflow Dagster pipeline deduping and enriching CRM leads
Data quality framework (Great Expectations) with auto-fail
FluentBit → S3 → Athena log pipeline with partition pruning
Ingest messy CSVs to Parquet lake with schema validation
Feature store (Feast Tecton) with point-in-time correctness
ELT reconciling Stripe and Shopify revenue nightly
Real-time IoT anomaly detection with Flink Spark
Data contract enforcement with JSON Schema + Pydantic
REST API gateway normalizing HubSpot, Salesforce, Notion
GraphQL facade over heterogeneous internal microservices
API versioning with backward-compatible deprecation
OAuth2 proxy unifying Google, Microsoft, GitHub SSO
Webhook relay with retry, DLQs, and idempotency keys
Rate-limiting and quota middleware for internal APIs
Bi-directional Airtable ↔ Postgres sync with conflicts
Connector for niche SaaS into data warehouse
Event-driven integration using NATS JetStream
Public API for internal tool with OpenAPI spec
MCP server for vector DB with typed tool schemas
MCP server exposing Jira Confluence to LLM tools
MCP server for Slack, Gmail, Calendar assistant agent
MCP server for Zendesk with auto-triage tools
MCP server wrapping internal pricing engine, audited
MCP server for BI tool enabling NL queries
MCP server for secrets manager with least-privilege
MCP server for document store with access control
MCP server for feature flags with rollout tools
MCP server for monitoring stack with incident tools
Basic RAG over internal docs with retrieval metrics
Multi-tenant RAG with isolation and access control
RAG with hybrid search and query rewriting
Agentic RAG with tool use and plan-and-execute
RAG with citations and grounded answers
RAG eval harness (recall, precision, faithfulness)
Retrieval with re-ranking for quality
LLM data doc generator from schemas and queries
Fine-tune small LLM for domain Q&A (e.g., SOX)
LLM agent drafting SQL with schema grounding
End-to-end RAG eval with golden datasets
Drift detection for embeddings and Recall@K
LLM response quality scorer (rules + model)
Cost attribution dashboard for LLM calls
Latency SLO dashboards for LLM endpoints
Canary eval suite for prompt changes (A B)
Offline online metric correlation for LLM changes
Red-team harness for jailbreaks and injection
Hallucination detector using claim verification
Feedback loop UI to flag bad answers, retrain
Sync identities Okta ↔ internal RBAC, audited
SOX-compliant change workflow for data pipelines
Integrate Snowflake with governance catalog, policies
Data access request portal (GDPR CCPA)
ServiceNow incidents ↔ Slack alerts, auto-runbooks
DLP policies across GDrive, S3, Postgres centralized
PII detection and masking for data exports
Secrets rotation in CI CD using Vault
Compliant audit trail for LLM prompts outputs
Model registry with deployment gates and sign-offs
CLI scaffolding microservice with lint tests CI
CLI for data contract validation on datasets
CLI running local LLM eval suites with diffs
CLI generating DB migrations from ORM safely
CLI managing feature flags across environments
CLI snapshot restore MCP configs per env
CLI profiling slow SQL with IndexAdvisor hints
CLI scraping API docs into searchable KB
CLI generating OpenAPI from live services
CLI rotating credentials via Vault APIs
Streaming ETL enriching clickstream 1s
Real-time fraud scoring microservice with inference
Real-time order funnel dashboard with WebSockets
Live chat analytics with topics and sentiment
Real-time log anomaly detector creating tickets
Streaming joins IoT telemetry with reference data
Real-time personalization updating per session
Live leaderboards with rank via Redis sorted sets
Real-time data quality monitors with Slack alerts
Real-time capacity forecasting for API endpoints
Attribute-based access control (ABAC) for APIs
Policy-as-code engine (OPA Rego) for data access
Data lineage tracker ingestion to ML features
Secrets scanning pre-commit hook with vault
Threat model generator from architecture diagrams
PII inventory scanning DBs and buckets
Compliant data retention and deletion scheduler
Encryption-at-rest with KMS key rotation
Fine-grained LLM output filtering for sensitive topics
Audit log viewer with immutable storage
Finance copilot reconciling invoices, flagging anomalies
Internal support bot triaging tickets with tools
Sales copilot enriching leads, drafting outreach
Ops runbook executor responding to alerts
HR onboarding bot provisioning accounts, tracking
Compliance copilot mapping controls to evidence
Data catalog explorer with NL search and lineage
Executive dashboard answering NL KPI questions
Customer health scoring with churn prediction
Incident assistant aggregating logs, metrics, traces