The debate between monoliths and microservices often frames scalability as a simple matter of performance. In reality, it extends to the challenge of growing code and teams together. In complex data and AI pipelines, orchestration, serverless components, and external foundation models—rapidly changing and subject to strict operational constraints—make it increasingly difficult to maintain a balance between maintainability, performance, and fast project onboarding.

In this talk, I will present an architectural approach that I have refined over time, which I call the micromonolith architecture. It is a model explicitly designed for the Python ecosystem that combines: • the cohesion, maintainability, and ease of onboarding typical of a monolith • the scalability and isolation of microservices

The architecture is built around serverless components common in data pipelines, clear development processes, and a standardized design of the Python modules in the repository, achieved through: • standardized scaffolding • minimal and functional flow documentation • systematic use of the Factory, Strategy, and Singleton patterns • standardized data exchange via a data lake • a workflow orchestrator (such as AWS Step Functions)

This approach enables teams to: • collaborate effectively, reducing conflicts and maximizing parallel work • keep core logic in a single repository per data pipeline, preserving the monolithic development experience • accelerate onboarding and support team growth • facilitate developers moving between different data pipelines • faithfully reproduce cloud behavior locally

This architectural paradigm helps teams grow, accelerates learning, and maintains agility even as pipelines become more complex and the technological landscape evolves rapidly.