Chafiq Madkour — Writing

Chafiq Madkour — Writing https://chafiqmadkour.com/en/writing/ Technical essays and notes on Spark, Databricks, orchestration and data quality. en-US Wed, 12 Aug 2026 09:00:00 +0000 Oozie → Airflow: what does not translate https://chafiqmadkour.com/en/writing/oozie-vers-airflow.html https://chafiqmadkour.com/en/writing/oozie-vers-airflow.html Wed, 12 Aug 2026 09:00:00 +0000 Orchestration An Oozie coordinator waits for data, an Airflow DAG fires on a clock: the whole migration sits in that gap. Spark 3 and old dates: the proleptic calendar trap https://chafiqmadkour.com/en/writing/spark3-dates-anciennes.html https://chafiqmadkour.com/en/writing/spark3-dates-anciennes.html Tue, 28 Jul 2026 09:00:00 +0000 Spark Spark 3 changed calendar: on old dates the same code returns a different value without raising an error. A row set aside in a regulatory filing is an under-declaration https://chafiqmadkour.com/en/writing/ligne-ecartee-sous-declaration.html https://chafiqmadkour.com/en/writing/ligne-ecartee-sous-declaration.html Tue, 14 Jul 2026 09:00:00 +0000 Data quality In a regulatory chain, an inner join does not clean the data: it removes rows you were required to report. AQE, DPP, skew join: what Spark 3 really fixes https://chafiqmadkour.com/en/writing/aqe-dpp-skew-join.html https://chafiqmadkour.com/en/writing/aqe-dpp-skew-join.html Thu, 18 Jun 2026 09:00:00 +0000 Spark AQE buys back bad estimates, not bad modelling: what it does, under which conditions, and what remains to rewrite. Java 8 → 17 under Spark: the list of surprises https://chafiqmadkour.com/en/writing/java8-java17-spark.html https://chafiqmadkour.com/en/writing/java8-java17-spark.html Wed, 10 Jun 2026 09:00:00 +0000 Platform Moving a Spark job from Java 8 to 17: the module system, the garbage collector, and the surprises that appear at runtime. Introducing TDD in a team that has never done it https://chafiqmadkour.com/en/writing/introduire-tdd-equipe.html https://chafiqmadkour.com/en/writing/introduire-tdd-equipe.html Fri, 22 May 2026 09:00:00 +0000 Method Going from 0 to 95 % coverage: start by pinning existing behaviour rather than by asking for tests. VaR: the shape of the computation comes before the optimisation https://chafiqmadkour.com/en/writing/var-forme-du-calcul.html https://chafiqmadkour.com/en/writing/var-forme-du-calcul.html Thu, 14 May 2026 09:00:00 +0000 Market finance On a VaR engine the volume is not what you read but what you generate: where the cost actually sits. Building a payments lakehouse alone: what it forces you to settle https://chafiqmadkour.com/en/writing/lakehouse-paiement-seul.html https://chafiqmadkour.com/en/writing/lakehouse-paiement-seul.html Thu, 16 Apr 2026 09:00:00 +0000 Architecture Building a payments and fraud lakehouse alone: the architecture decisions that held, and why.