কোর্সের চূড়ান্ত পর্যালোচনা
এই পাঠে যা শিখবেন
- প্রতিটি module-এ যা শিখলেন তার consolidation — connecting the dots
- ৩-৬-১২ মাসের next-step roadmap — কী বানালে portfolio আকর্ষণীয়
- Bangladesh-এ Data Engineer career — salary, role progression, hiring landscape
- "Senior" data engineer-এর mindset — যা বইয়ে নেই
১ · যা শিখেছি — module-wise pull-together
এই কোর্স একটি journey ছিল — "ETL কী?" থেকে শুরু করে production fraud detection pipeline পর্যন্ত। প্রতিটি module ভিন্ন একটি foundation দিয়েছে। চলুন consolidate করি।
M1 (L01-07): Foundations — পাইপলাইন কেন, OLTP/OLAP, lake/warehouse/lakehouse, ETL/ELT, schema, SQL.
M2 (L08-15): Batch & orchestration — Postgres, MongoDB, Spark, Airflow, dbt.
M3 (L16-21): Streaming & real-time — Kafka, Spark Streaming, Flink, CDC.
M4 (L22-29): Cloud & production — Snowflake, BigQuery, Iceberg/Delta, quality, governance, cost, project.
২ · মডিউল ১ recap — Foundations (L01-L07)
মূল প্রশ্ন: "কেন আলাদা DE role? Backend dev কি করতে পারে না?"
- L01-02: OLTP (Postgres-এ live transaction) ও OLAP (warehouse-এ analytics) — fundamentally different. Single DB-এ both impossible at scale.
- L03: Data Lake (cheap, raw) → Warehouse (structured, expensive) → Lakehouse (best of both — Iceberg/Delta).
- L04: ETL (transform before load) vs ELT (load raw, transform in warehouse) — modern stack ELT-favored.
- L05: Star schema (fact + dim, query-friendly) vs Snowflake (normalized, storage-efficient).
- L06-07: SQL = DE-এর mother tongue. JOIN, CTE, window functions — ad-hoc থেকে production সব জায়গায়।
৩ · মডিউল ২ recap — Batch & orchestration (L08-L15)
মূল প্রশ্ন: "ডেটা কীভাবে নিয়মিত process & schedule করব?"
- L08-09: PostgreSQL (relational, ACID, OLTP champion) vs MongoDB (document, flexible schema, read-heavy).
- L10-12: Apache Spark — distributed compute. RDD vs DataFrame vs Dataset. Catalyst optimizer, partition tuning, broadcast join.
- L13-14: Airflow — DAG, scheduler, executor (Celery/Kubernetes), XCom, sensors, hooks.
- L15: dbt — analytics engineering. SQL + Jinja + version control. Modular, testable, documented warehouse models.
একসাথে যা পেলেন:
- Source → Spark batch read → cleaned write → Airflow orchestrate → dbt model → BI. Daily refresh, idempotent, observable.
- Bangladesh context: bKash-এ এই stack-ই production — Airflow + Spark + warehouse.
৪ · মডিউল ৩ recap — Streaming & real-time (L16-L21)
মূল প্রশ্ন: "Batch ২৪-hour latency-তে যেসব decision হারায় — কীভাবে real-time-এ?"
- L16: Streaming-এর fundamentals — event-time vs processing-time, watermark, windowing, late data.
- L17-18: Kafka — producer/consumer/topic/partition/offset. Replication factor, retention policy, exactly-once semantics.
- L19: Spark Structured Streaming — micro-batch ১০০ms-2s latency, batch-stream code unification.
- L20: Apache Flink — true streaming, ১০-১০০ms, complex stateful processing.
- L21: CDC (Change Data Capture) — Debezium pattern. OLTP → stream → analytical platform.
Streaming-এর use cases:
- Fraud detection (Daak Pay-এর মতো) — <১ সেকেন্ড decision.
- Real-time dashboard — Pathao operations team driver allocation.
- Personalization — Daraz homepage live update.
- IoT — Robi tower telemetry, smart meter.
- Trigger workflows — bKash agent low-balance auto-restock alert.
৫ · মডিউল ৪ recap — Cloud & production (L22-L28)
মূল প্রশ্ন: "MVP থেকে production-grade system হতে কী কী matter করে?"
- L22-23: Snowflake (multi-cloud, separate compute/storage, virtual warehouses) vs BigQuery (serverless, slot-based, GCP-native) vs Redshift (AWS-native, RA3 nodes).
- L24: Delta Lake (Databricks) ও Iceberg (open) — table format. ACID on lake, time travel, schema evolution.
- L25: Data quality — Great Expectations, dbt tests, Soda. "trust but verify".
- L26: Governance & lineage — DataHub/Amundsen/OpenMetadata. Catalog, ownership, classification, access.
- L27: Cost optimization — partition, cluster, materialized view, FinOps culture.
- L28: Capstone — full pipeline integration, real-world Daak Pay fraud detection.
৬ · "Senior" mindset — যা বইয়ে নেই
Senior data engineerSenior Data Engineerসাধারণত ৪+ বছর experience, multiple production system-এ ownership, junior mentorship, ও cross-functional collaboration. Bangladesh-এ ৫০-১২০K BDT/মাস।-এর কাজ "Spark code লেখা" না — system design, trade-off navigation, business understanding.
- Trade-off navigator: "Snowflake বনাম BigQuery" — context ছাড়া answer ভুল।
- Failure-thinker: "এই pipeline ভেঙে গেলে কী হবে?" — pre-mortem, runbook.
- Cost-conscious: $৫,০০০/মাস extra = engineer salary. Optimization business-impact.
- Communication bridge: CFO-কে SQL explain না, business outcome explain.
- Documentation discipline: "Future-me & future-team" thinking.
- Mentor mindset: Junior-এর code review = team scaling.
৭ · ৩-৬-১২ মাসের next-step roadmap
৩ মাস — portfolio building:
- L28-এর Daak Pay pipeline pure local Docker Compose-এ run করুন। GitHub-এ push.
- একটি Bangladesh public dataset (BBS, BTRC) দিয়ে end-to-end project — ingest → transform → dashboard.
- Personal blog (Medium/Dev.to) — ৫টি technical post. "How I built X with Airflow".
- Apache Iceberg + DuckDB local lakehouse — modern stack hands-on.
- Snowflake free trial বা BigQuery sandbox — কাজে লাগিয়ে query optimization technique post.
৬ মাস — production exposure:
- Open-source contribution — Airflow provider, dbt package, Iceberg connector. Even small docs PR valuable.
- Cloud certification — AWS Data Engineer Associate, GCP Professional Data Engineer. বিশ্বে recognized.
- Internship/junior role — bKash, Pathao, Daraz, Robi, Brain Station 23, ShareTrip — সবাই DE নিচ্ছে।
- Real production pain experience — incident response, on-call rotation, post-mortem.
- Stack proficiency depth — যেকোনো একটিতে expert (Spark বা Airflow বা dbt).
১২ মাস — specialization:
- একটি specialty বেছে নিন — streaming (Flink), governance (DataHub), cloud (multi-cloud architect), ML platform (Feast + MLflow).
- Conference talk বা meetup — Dhaka Tech Talks, GDG Dhaka.
- Mentor junior — internal team বা community.
- Senior role transition — current org-এ senior, বা senior offer-এ switch.
৮ · MLOps — most natural next course
Data Engineering pipeline তৈরি করেছেন — সেই data দিয়ে ML model train, deploy, monitor কীভাবে? এটাই MLOps.
- Model lifecycle management — MLflow, Weights & Biases, ZenML.
- CI/CD for ML — GitHub Actions, Argo Workflows, Kubeflow Pipelines.
- Feature store production — Feast, Tecton.
- Model serving — TensorFlow Serving, Triton, BentoML, KServe.
- Monitoring — drift detection, A/B testing, shadow mode.
- Governance — model card, fairness audit, explainability.
Data Engineer + MLOps = একটি rare combo যা Bangladesh-এ ১,০০,০০০+ BDT/মাস commands. ABCL TECH-এ পরবর্তী course এই path.
৯ · বিকল্প path — Data Science vs DevOps vs Backend
| Path | Strength leverage | কী লাগে নতুন | BD demand |
|----------------|---------------------------|-----------------------------|---------------|
| MLOps | Pipeline + infra | ML lifecycle, model serving | ⭐⭐⭐⭐⭐ |
| Data Science | SQL + Python + statistics | ML algos, business savvy | ⭐⭐⭐⭐ |
| Analytics Eng | dbt + SQL + business | Stakeholder communication | ⭐⭐⭐⭐ |
| Platform Eng | Airflow + K8s + cloud | DevOps, observability | ⭐⭐⭐⭐ |
| Backend → DE | OLTP knowledge | Distributed systems | ⭐⭐⭐ |
| Data Architect | Big-picture trade-offs | Senior design experience | ⭐⭐⭐⭐ (Sr) |
| ML Engineer | Pipeline + math | Deep ML, optimization | ⭐⭐⭐⭐ |১০ · Bangladesh-এ Data Engineer market — ২০২৫ outlook
Demand পক্ষ:
- Fintech (top hirers): bKash, Nagad, Rocket, Upay, Tap'n'Pay, Pathao Pay. Fraud, risk, lending — সবার DE team scaling.
- E-commerce: Daraz, Pickaboo, Chaldal, Foodpanda, Sheba.xyz. Personalization, supply chain, COD optimization.
- Telecom: Grameenphone, Robi, Banglalink. CDR analytics, churn, network optimization.
- Banking: Eastern Bank, Dutch-Bangla, City Bank, BRAC Bank. AML, regulatory reporting, customer 360.
- Ride-sharing/logistics: Pathao, Uber Bangladesh, Steadfast, RedX. ETA, demand forecasting.
- Software services: Brain Station 23, Tiger IT, Therap, Kaz Software — international client-এর জন্য DE.
- Startups: ShareTrip, Bondstein, Truck Lagbe, Maya — series A/B funding-এ data team build.
Salary range (২০২৫ estimate, BDT/month gross):
- Junior DE (০-২ বছর): ৩০-৬০K.
- Mid DE (২-৪ বছর): ৬০-১২০K.
- Senior DE (৪-৭ বছর): ১২০-২৫০K.
- Lead/Staff DE (৭+ বছর): ২৫০-৫০০K+ (top fintech, multinational).
- Remote international (US/EU client): $৩,০০০-৮,০০০/month.
Skill premium:
- Cloud certification (AWS/GCP) — ১৫-২০% premium.
- Streaming expertise (Kafka, Flink) — ২০% premium.
- Governance + compliance (BB-regulated industry) — ১০-১৫% premium.
- English communication — international remote unlocked.
১১ · "যা শিখলেন না কিন্তু শিখতে হবে"
এই কোর্স সব cover করে নি। নিচের topics — production-এ critical, পরের পদক্ষেপে শিখুন:
- Container & orchestration: Docker, Kubernetes, Helm — DE-এর deployment মেরুদণ্ড।
- Linux & networking: bash, TCP/IP, DNS, load balancer — debugging foundation.
- Terraform/IaC: infrastructure as code — production-এ click-ops disaster.
- Observability: Prometheus, Grafana, OpenTelemetry, ELK/Loki — monitoring depth.
- Security: IAM, KMS, secrets management, network security.
- System design interview: 1-hour-এ "design Twitter feed" বা "design ride matching" — DE perspective.
- Distributed systems theory: CAP, consistency models, consensus (Raft/Paxos basics).
- Software engineering practice: testing, code review, modular design — DE-তে underemphasized.
১২ · Final words — যা মনে রাখবেন
Data Engineering — একটি craft, একটি engineering discipline. এর সবচেয়ে valuable lesson — "good enough today" beats "perfect next year". Production-এ ship করা, observe করা, iterate করা — এটাই rhythm.
৭টি principle যা senior DE মনে রাখেন:
- Idempotency — পুনরায় চালালে same outcome. Replay safety.
- Observability — monitor first, optimize later.
- Schema discipline — contract testing, breaking change explicit.
- Lineage by default — debugging cost-saver.
- Cost as feature — bill = engineering metric.
- Compliance early — retrofit nightmare.
- Empathy — analyst, scientist, business — তারা DE-এর customer.
১৩ · MLOps কোর্সে স্বাগতম
পরবর্তী natural step — MLOps. Pipeline দিয়ে data ready, এখন সেই data দিয়ে model production-এ। L28-এ যে fraud detection pipeline দেখলেন — তার model layer (MLflow + FastAPI + monitoring) — সেটাই MLOps-এ গভীরভাবে শিখবেন। নিচের "MLOps কোর্সে যান" link থেকে শুরু করুন। শুভকামনা!
ভাবনার প্রশ্ন
প্রতিটি প্রশ্ন নিজে কিছুক্ষণ ভাবুন — তারপর "→ উত্তর" চাপুন।
প্র ০১ আপনি ৬ মাস DE শিখেছেন। কোম্পানি interview-এ technical round-এ প্রস্তুতি কেমন হবে? কোথায় focus সবচেয়ে বেশি?
DE interview pattern — প্রায়ই multi-stage, technical broad, design-heavy. বাংলাদেশ-এ top company (bKash, Pathao, Daraz)-এ ৩-৪ round common.
Round 1 — SQL screening:
- ৪-৫টি SQL problem, increasing difficulty.
- Window functions, CTE, recursive query — must-know.
- "Top 3 customers per category" — RANK + PARTITION classic.
- "Find users who placed order in 3 consecutive months" — gap-and-island technique.
- Resource: LeetCode SQL hard, StrataScratch.
Round 2 — System design:
- "Design Daraz-এর order analytics platform" — open-ended.
- Approach: clarify requirement → propose architecture → discuss trade-offs → handle scale.
- Diagram skills — boxes, arrows, latency/cost annotations.
- L28-এর Daak Pay capstone-এর sketch your interview answer.
- Resource: "Designing Data-Intensive Applications" by Kleppmann.
Round 3 — Tool deep-dive:
- "Spark partition skew কী, কীভাবে detect & fix?"
- "Airflow XCom-এ large data পাঠানো অনুচিত কেন?"
- "Kafka exactly-once-এর ৩টি condition কী?"
- একটি tool depth — Spark বা Airflow বা Kafka.
Round 4 — Behavioral + project walk:
- STAR format — situation, task, action, result.
- Past project deep-dive — "কেন Spark, Flink না?"
- Conflict resolution — "Stakeholder unrealistic deadline চাইল, কী করেছেন?"
- Failure story — "Production-এ ভেঙেছে, post-mortem-এ কী শিখলেন?"
Focus area priority:
- (১) SQL: ৩০% interview weight. ৫০-৭০ medium-hard problem solve.
- (২) System design: ২৫%. ১০টি classic problem framework মুখস্থ — feed, ride matching, fraud, recommendation, search.
- (৩) Project narrative: ২০%. ১-২টি real project crisp explanation.
- (৪) Tool depth: ১৫%. Spark + Airflow + dbt — production scenario.
- (৫) Behavioral: ১০%. 8-10 STAR story prep.
Common mistake:
- "আমি Tool X জানি" — but couldn't explain when to NOT use.
- System design-এ scale handle না করে only happy path.
- Project-এ "team did" — but couldn't articulate own contribution.
- SQL window function attempt-ই করেনি — automatic rejection.
Bangladesh-specific:
- Local context-এ comfortable — bKash, Daraz scale references.
- Bengali-English mix communication — natural switch.
- Compliance awareness — BB regulation, AML basics.
- Cost-consciousness — startup budget, runway impact.
মূল কথা: Interview = signaling. ছ্যাঁকা SQL, framework-driven design, crisp project narrative — three-legged tripod. Practice mock interviews সমান-skill peer-এর সাথে। Pramp, interviewing.io, internal study group. Theory ছাড়াই reps. ৬ মাস serious prep — top company offer realistic.
প্র ০২ "Junior DE" থেকে "Senior" — Bangladesh-এ ৩-৪ বছরের journey-তে কী matter সবচেয়ে বেশি? Hard skill, soft skill, দু'টার balance?
এই transition tech industry-র সবচেয়ে nuanced. ৩ বছরে সবার "senior" title আসে না — কিছু engineer ৭ বছরেও mid-level.
Year 1 — Mastery of one stack:
- একটি tool-এ deep ownership — Airflow + Spark + warehouse.
- Production ticket, on-call shift, post-mortem participation.
- Daily output — pipeline ship, bug fix, query optimize.
- Mentorship request — daily-stand up question, weekly 1:1 manager.
- Hard skill: ৭০% time, soft skill: ৩০%.
Year 2 — Cross-stack & ownership:
- Adjacent stack expand — streaming pick up, dbt deep-dive.
- Project lead — small initiative end-to-end.
- Documentation — runbook, design doc, RFC.
- Junior onboarding — buddy role.
- Cost optimization initiative — measurable savings.
- Hard skill: ৬০%, soft skill: ৪০%.
Year 3 — System design & influence:
- Architecture decision — RFC author, multiple stakeholder align.
- Cross-team collaboration — DS, backend, product.
- Open-source contribution বা talk বা blog.
- Hiring loop participation — interview design.
- Strategic thinking — "next year-এ team কী করবে"।
- Hard skill: ৫০%, soft skill: ৫০%.
Year 4 — Senior signals:
- Trusted with ambiguity — vague problem-এ approach define.
- Multi-quarter project ownership.
- Junior promotion sponsorship.
- Manager-replacement signal — manager out হলে continuity.
- External recognition — community, blog readership, conference.
- Hard skill: ৪০%, soft skill: ৬০%.
Hard skill milestones:
- SQL — fluent, optimization-aware.
- One distributed system internals understand (Spark বা Kafka).
- System design 5+ pattern reproducible.
- Cloud platform certified expert (AWS/GCP/Azure).
- One specialty depth (governance বা streaming বা ML platform).
Soft skill milestones:
- Written communication — clear PR description, design doc.
- Verbal — present to non-technical stakeholder.
- Conflict navigation — disagree without burning bridge.
- Time management — multiple project juggle.
- Mentorship — junior-এর growth visible.
Bangladesh-specific accelerator:
- English communication — international client/team unlock. ১.৫x salary potential.
- Public profile — LinkedIn, GitHub, Twitter active. Recruiter inbound.
- Community engagement — Dhaka tech meetup, GDG.
- Side project — open-source contribution, freelance international.
Antipatterns:
- "Senior means alone" — opposite, senior multiplies team.
- "Senior means perfect code" — opposite, senior makes good trade-offs.
- "Senior means say yes to everything" — opposite, senior prioritizes ruthlessly.
- "Senior comes from years" — only if the years are intentional growth.
মূল কথা: Senior একটি capability, not a calendar event. Hard skill table-stakes, soft skill differentiator. Bangladesh-এ talented junior অনেক, true senior কম — যিনি consistently hard + soft balance করেন, তিনি rare. সেই rarity-ই compensation premium.
প্র ০৩ আপনি একটি Bangladesh startup-এ একমাত্র DE. ৬ মাসে data platform 0-to-1 build করতে হবে। কী strategy? কী trap এড়াবেন?
Solo DE in startup — career-defining opportunity. Right execution = ৬ মাস পর senior DE. Wrong = burnout + rebuild.
Month 1 — Discovery + foundation:
- Stakeholder interview — CEO, CTO, head of product, marketing, finance. কে কী data ব্যবহার করছে?
- Current state audit — কোন kind of data, কোথায়, কে দেখছে। Spreadsheet sprawl probably.
- Top 3 use case identify — fraud, retention, growth — যা পেলে business decision-এ impact.
- Tech selection — bias toward managed (BigQuery, Snowflake), open source where possible.
- Quick win — একটি critical dashboard ship করুন, credibility build.
Month 2-3 — Core pipeline:
- Source → warehouse — ingest pipeline (Fivetran/Airbyte managed, custom Python only if needed).
- dbt project setup — ১০-১৫ core model, business-validated.
- BI tool — Metabase (free) or Looker Studio. Stakeholder self-serve.
- Monitoring basics — pipeline health Slack notification.
- Documentation — README, runbook from day one.
Month 4-5 — Polish + scale:
- Quality testing — Great Expectations or dbt tests.
- Catalog — DataHub light বা just dbt docs.
- Cost monitoring — daily alert, optimization pass.
- Onboarding doc — next DE join করলে day-1 productive.
- Senior stakeholder weekly review — trust + visibility build.
Month 6 — Roadmap + hire:
- Year-2 roadmap propose — streaming, ML support, governance.
- Hire pitch — "next DE হিরে" — JD draft, interview loop design.
- Achievement summary — metrics, savings, dashboards-shipped.
Trap-গুলো:
- (১) Tool tourism: "Airflow + Dagster + Prefect সবগুলো try করি" — ১টি pick এবং deeply learn.
- (২) Bleeding-edge bias: Iceberg + DuckDB + Polars — startup-এ stable boring stack better.
- (৩) Premature scale: "5 million daily user-এর জন্য design" — startup-এ ৫০০ user; over-engineer disaster.
- (৪) Vendor lock-in trauma: Snowflake/BigQuery অসাধারণ — "lock-in" fear-এ self-host Postgres-এ ১০x effort.
- (৫) Single-handedness fatigue: Solo DE 24/7 ownership. On-call boundary, vacation OOO — early establish.
- (৬) No documentation: "I'll write later" — bus factor 1. Disaster.
- (৭) Stakeholder neglect: Engineering polish but business case absent — fired in Q3.
- (৮) Perfect data syndrome: "Data ১০০% clean না হলে dashboard ship করব না" — never ships.
Mindset shift:
- "Engineering excellence" → "business outcome".
- "My code" → "team asset".
- "Solve perfectly" → "iterate continuously".
- "Hide complexity" → "pair-program with stakeholder".
Bangladesh-specific advantage:
- Founder generally accessible — direct conversation possible.
- Domain context easy — local market, banking, mobile money intuitive.
- Compliance opportunity — BB-regulated industry, governance experience valuable.
- Mentor network — Tech-Bangla, BJIT alumni, ShareTrip ecosystem — reach out.
মূল কথা: Solo DE startup = compressed ৩ বছরের senior experience. ৬ মাসে ১২ months work, কিন্তু careful — burnout danger. Boundaries, clarity, communication — equal weight to technical skill. Successful execute করতে পারলে — যেকোনো post-startup role open.
প্র ০৪ ২০২৫-৩০ — Bangladesh-এ Data Engineering field কীভাবে evolve করবে? Generative AI, agents, automation — DE role বদলে যাবে নাকি?
Hard prediction — কিন্তু trend identifiable. Engineer-এর জন্য planning matters.
Bangladesh-specific drivers:
- Smart Bangladesh ২০৪১: সরকারের vision — সব ministry data-driven. DE government project-এ huge demand.
- Fintech consolidation: bKash, Nagad, Rocket scale-up; international expansion (e.g. Pakistan, Nepal).
- Bank digitization: Eastern Bank, BRAC Bank — legacy core banking → cloud modernize.
- Data Protection Act enforcement: compliance role surge — DPO, governance specialist.
- Foreign client export: US/EU client Bangladesh-এ data team — remote DE rate increase.
- BTRC 5G rollout: IoT data explosion.
Technology trend:
- Modern data stack consolidation: "best of breed" tool plethora → unified platform (Databricks, Snowflake all-in-one).
- Lakehouse domination: Iceberg/Delta open standard winning. Vendor neutrality ahead.
- Streaming first: Batch decline; real-time default expectation.
- SQL renaissance: dbt + analytics engineer separating "analytics work" from "data engineering".
- ML/AI infrastructure: Feature store, vector DB, embedding pipeline — DE territory expand.
- Multi-cloud common: single-cloud lock-in fear, multi-cloud reality. Skill demand.
GenAI impact on DE role:
- Augmentation, not replacement: GitHub Copilot, ChatGPT — boilerplate দ্রুত। Critical thinking, design judgment — still human.
- SQL generation: natural language → SQL — analyst self-serve increase, DE infrastructure focus.
- Pipeline scaffolding: "create dbt model for orders" — boilerplate AI generated, DE customize.
- Documentation auto-generation: dbt docs + LLM = lineage stories explained.
- Anomaly detection: AI-driven monitoring tool — Anomalo, Monte Carlo style.
- Code review: AI first-pass, human deep review.
New roles emerging:
- AI/ML Data Engineer: embedding pipeline, vector DB management, RAG infrastructure.
- Data Reliability Engineer (DRE): SRE applied to data; SLA, on-call.
- Data Product Manager: internal data product owner — analytics engineer + PM hybrid.
- Privacy Engineer: tokenization, differential privacy, consent management.
- FinOps Engineer: cloud cost expert — full-time role.
Skill durability ranking (2025-30):
- High durability (will matter): SQL, system design, distributed systems concept, business judgment, communication.
- Medium durability: specific tools (Spark, Airflow) — survive but evolve.
- Low durability: tool-specific tricks, vendor-specific UI — replaced rapidly.
- Rising: ML/AI infrastructure, governance, privacy, cost engineering.
Bangladesh advantage opportunity:
- Time zone — US/EU overlap, remote-friendly.
- Cost arbitrage — international rate Bangladesh-এ 2x local.
- Domain expertise — emerging market fintech world-class understanding.
- English fluency growing — communication barrier reducing.
- Open-source contribution — local engineer global recognition.
Risk for engineers:
- "SQL writer" only — automation eats lunch.
- Tool fanatic — stack pivots leave behind.
- No business context — technical-only specialist limited.
- Localized only — global market participation missed.
Action items for the next 5 years:
- Pick a specialty — don't be "Jack of all trades, master of none".
- Build public portfolio — GitHub, blog, talks.
- Connect globally — Twitter, LinkedIn, Reddit, conference.
- Mentor + be mentored — both directions.
- Solve real Bangladesh problems — local impact + visibility.
- Keep learning — quarterly new skill addition.
মূল কথা: Field expanding, not contracting. Tools change, fundamentals don't. Adaptability + critical thinking + business sense = career insurance. Bangladesh-এ Data Engineer-এর জন্য next ৫ বছর — historic opportunity. যিনি foundation শক্ত রেখে continuously evolve — তিনি leader হবেন। ABCL TECH-এ আপনার যাত্রা শুরু হলো — বাকিটা আপনার হাতে।
অনুশীলন
-
Self-assessment: ২৯ পাঠের প্রতিটি topic-এ নিজেকে rate করুন (1-5). কোন ৩টি topic-এ আপনি সবচেয়ে দুর্বল? পরের ১ মাসে সেই ৩টি-তে hands-on practice plan তৈরি করুন।
Self-assessment template:
- SQL (window, CTE, recursive) — 1-5
- Spark internals (partition, shuffle, broadcast) — 1-5
- Airflow (DAG design, executor, sensors) — 1-5
- Kafka (partition, offset, exactly-once) — 1-5
- dbt (model materialization, test, doc) — 1-5
- Warehouse (Snowflake/BigQuery) — 1-5
- Streaming (Spark Streaming, Flink) — 1-5
- Iceberg/Delta — 1-5
- Governance (catalog, lineage) — 1-5
- Cost optimization — 1-5
Action plan example for weakest 3:
Suppose Spark = 2, Kafka = 2, Iceberg = 1. Then:
- Week 1-2: Spark — Daniel Abadi book chapter, Databricks free training, Spark-এ ৫টি optimization exercise.
- Week 3: Kafka — Confluent free course, single-broker setup, producer/consumer Python script.
- Week 4: Iceberg — DuckDB + Iceberg local lakehouse, time travel + schema evolution try.
-
Portfolio project pitch: একটি Bangladesh-context project propose করুন — ১-page outline (problem, dataset, stack, deliverable).
Example: BDT Stock Market Analytics Platform
- Problem: DSE (Dhaka Stock Exchange) retail investor-দের real-time analytics absent. Free, open data dashboard build.
- Dataset: DSE end-of-day prices (public CSV), historical 5-year, 600+ scrip.
- Stack:
- Ingest: Python scraper + GitHub Actions cron.
- Storage: DuckDB + Iceberg (local lakehouse).
- Transform: dbt models — moving averages, sector aggregates, volatility.
- Orchestrate: GitHub Actions (cheap).
- Serve: Streamlit dashboard, deployed on Streamlit Cloud (free).
- Monitor: dbt test + Slack notification.
- Deliverable: public GitHub repo, deployed dashboard, Bengali blog post explaining.
- Why impressive: end-to-end DE skill demonstration, local impact, $0 cost runs forever.
- Resume bullet: "Built end-to-end DSE analytics platform serving 1000+ retail investors monthly — Iceberg, dbt, Streamlit stack with $0 infra cost."
-
Career conversation: পরের ২ মাসে ২ জন senior DE-এর সাথে coffee chat / LinkedIn message করুন। কী কী প্রশ্ন করবেন?
Reach-out template (LinkedIn DM):
"হ্যালো [Name], আমি ABCL TECH-এর Data Engineering কোর্স সদ্য complete করেছি। আপনার [Company]-এ DE journey সম্পর্কে আমি ১৫-২০ মিনিটের একটি conversation করতে চাই। আপনার সুবিধা মতো — Zoom/coffee যেকোনোটাই কাজ করবে। ধন্যবাদ।"
Questions to ask:
- "আপনার career journey — কীভাবে junior থেকে senior?"
- "Day-to-day কী করেন? অনুমান বনাম reality?"
- "Past ১২ মাসে সবচেয়ে valuable skill কী শিখলেন?"
- "নতুন DE-কে কী advice দেবেন — যা আপনি early career-এ জানতে পারলে ভালো হত?"
- "Bangladesh-এ DE community কোথায় active? কোন meetup/group recommend?"
- "Hiring-এ আপনি কী খোঁজেন? Junior candidate-এ কী stand out করে?"
- "আমি একটি project plan করছি [explain briefly] — feedback দিতে পারেন?"
Follow-up: 1-week পর thank-you message + small update. 6-month পর "remember our chat — here's what I built". Network compounds.
আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ
- MLOps কোর্সে যান পরবর্তী course DE pipeline তৈরি — এখন সেই data দিয়ে ML model production-এ। Natural next step.
- পাঠ ২৮ · প্রজেক্ট: end-to-end ML pipeline আগের পাঠ Capstone project — সব শেখা একসাথে।
- Data Science কোর্স বিকল্প পথ DE foundation-এর উপর modelling depth — statistics, ML, business analytics.
- সব AI Courses দেখুন ABCL TECH Python, ML, DL, NLP, CV, GenAI, RL, MLOps — সব AI কোর্স একসাথে।