1. Introduction
After five years of accumulating software engineering experience alongside my machine learning and data science roles, I have noticed a common pattern: we do not consistently teach or practice solid planning, and we often build systems that are difficult to understand and communicate. The result is familiar: a big ball of mud, or spaghetti code.
Over the last two decades, teams have often absorbed this technical debt by adding engineers who build a mental model of the spaghetti and translate requirements into code. Another response is to split the code into microservices—or, in some cases, micro balls of mud—so different people can take responsibility for understanding different parts. Splitting the code does not, by itself, make the boundaries clearer.
I think we need two pillars to redress the balance: disciplined planning, with a shared way to describe the design and clear rules connecting that plan to what gets built; and processes that keep the implemented system understandable as it changes. Section 2 puts the relevant software engineering concepts and tools into a hierarchy. Section 3 turns to modeling and process guidance.
How does this apply to AI agents generating massive amounts of code? As code generation becomes cheaper, I can tolerate some low-level style issues or minor code smells. Personally, I no longer inspect every line of low-level code. But I think it is a massive strategic mistake to let AI agents loose to invent their own architecture without explicit boundaries or tests that check whether the implementation respects them. Cheap code makes architectural judgment more consequential.
2. Concepts and tools
Before discussing a development process, we need a shared vocabulary. Software engineering is the umbrella discipline: specifying, designing, building, testing, delivering, operating, maintaining, and eventually retiring software. The concepts beneath that umbrella answer different questions. The following is a map of their roles, not a rigid ladder in which each idea belongs to exactly one level.
2.1 A map of the concepts
- Principles guide decisions. They explain what to favor: simplicity, cohesive responsibilities, explicit dependencies, testability, and controlled coupling.
- Patterns describe reusable solution structures. They connect a recurring problem and context to a solution with known tradeoffs.
- Mechanisms make solutions work. Functions, modules, interfaces, composition, dependency injection, transactions, locks, and message passing provide concrete capabilities.
- Architecture governs consequential structure. It defines major boundaries, ownership, contracts, communication, and decisions that are costly or risky to reverse.
- Refactoring improves internal structure. It preserves intended externally observable behavior while making future changes easier or safer.
- Delivery moves verified changes toward users. It connects integration, repeatable builds, tests, artifacts, deployment, release, and recovery.
- Tooling provides feedback and automation. It helps us understand, create, validate, deliver, and operate software.
- Methodologies organize work; practices make it repeatable. Scrum or Kanban organize coordination, while code review, testing, and continuous integration are practices.
2.2 Principles, patterns, and mechanisms
KISS favors the simplest design that meets the need. YAGNI challenges speculative capability. DRY concerns duplicated knowledge, not merely repeated text. Separation of concerns, high cohesion, low coupling, encapsulation, and dependency inversion help us decide where responsibilities belong. SOLID collects related object-oriented principles. These guidelines can pull in different directions; applying them requires judgment rather than maximizing every principle independently.
Design patterns supply possible structures at the code level. Creational patterns such as Factory and Builder address construction; structural patterns such as Adapter and Facade organize interfaces and relationships; behavioral patterns such as Strategy, Observer, and Command organize interactions. A pattern is useful when its problem and tradeoffs match ours. A first-class function may express a replaceable strategy more clearly than a hierarchy of classes.
Mechanisms implement those choices. Dependency inversion is a principle; dependency injection is a way to apply it; constructor parameters or a container can provide the injection. Likewise, a scheduler is a mechanism, while the rule deciding when to retry a job is policy. Mixing these levels makes a design harder to explain and evaluate.
2.3 Architecture: boundaries, ownership, and evidence
Architecture spans domain capabilities, module dependencies, data ownership, runtime interactions, deployment, security, reliability, and team responsibility. A folder, repository, process, container, or database is not automatically a meaningful logical boundary. Start with behavior, business rules, consistency requirements, lifecycle, and reasons to change; then decide which boundaries also need separate deployment.
Architectural styles address different axes. Layers separate responsibilities; vertical slices organize use cases; a modular monolith preserves enforceable boundaries inside one deployment; ports and adapters isolate application policy from external technology. Event-driven communication and microservices add other choices. These styles can coexist. Distributing a tangled application can create a distributed monolith, with network and operational costs added to the original coupling.
Understanding a system also requires several views. The domain model describes business concepts; the design model describes intended structure; the code model describes implementation. Module, runtime, deployment, data, and security views answer different questions. We need explicit mappings between them, and a distinction between prescriptive models—what should exist—and descriptive models—what currently exists.
Contracts give the lines between components meaning: operations, data ownership, invariants, compatibility, authorization, ordering, failure, and retry behavior. Decision records preserve context, alternatives, assumptions, and tradeoffs. Architectural fitness functions—such as dependency rules, contract checks, latency measurements, or recovery exercises—test important commitments as the system evolves. A green check proves its encoded condition, not the entire architecture.
2.4 Cloud and distributed-system patterns
Cloud patterns are not a separate maturity level. They address conditions introduced by remote calls, concurrency, distributed state, variable load, and partial failure. A timeout does not prove that an operation failed; a message may arrive twice; a process may commit a change and crash before publishing its event. Select patterns for a concrete failure mode and quality requirement, with clear ownership and recovery behavior.
- Resilience: timeouts bound waiting; bounded retries with backoff and jitter address transient, safe-to-repeat work; circuit breakers reduce pressure on failing dependencies; bulkheads contain failure; rate limits and load shedding protect capacity. Retrying everything can amplify an outage.
- Messaging: queues smooth load, competing consumers share work, and publish/subscribe notifies independent consumers. Idempotent handlers make duplicates safe; dead-letter queues need alerts, an owner, and a replay process. A queued request is accepted for processing, not completed.
- State and consistency: a transactional outbox commits a state change and publication intent together. Sagas coordinate local transactions and compensating business actions; compensation is not a perfect rollback. Cache-aside trades freshness and invalidation complexity for faster reads. CQRS separates read and write models; event sourcing stores events as authoritative history. Neither automatically requires the other.
- Scale and performance: horizontal scaling adds instances, partitioning divides data or work, and materialized views precompute reads. Name the actual constraint—traffic, latency, data volume, or team coordination—before adding infrastructure.
- Deployment and evolution: rolling, blue-green, and canary approaches control how versions reach users. Strangler migration replaces legacy behavior incrementally; expand/contract lets old and new contracts coexist. Switching traffic back does not undo an incompatible data migration.
- Security and operations: gateways handle edge concerns, federated identity delegates authentication, and short-lived resource-scoped tokens can grant narrow access. Correlation identifiers, tracing, health endpoints, metrics, and alerts make behavior diagnosable. Liveness and readiness answer different questions.
These patterns work in combinations. Reliable event publication, for example, may need an outbox, an asynchronous publisher, at-least-once delivery, idempotent consumers, and visible recovery. That combination does not create a universal end-to-end exactly-once guarantee. Every added pattern has cognitive, operational, and financial costs.
2.5 Refactoring, delivery, and practices
Refactoring is behavior-preserving structural change. That behavior can include public interfaces, persisted meaning, authorization, emitted events, failure semantics, and required performance—not just returned values. Characterization tests help protect existing behavior while responsibilities and dependencies are improved. Moving code into more files, introducing a pattern, or raising a dashboard score is not an improvement by itself.
Continuous integration combines frequent integration with automated validation. Continuous delivery keeps accepted changes deployable; continuous deployment automatically promotes passing changes into production. A build creates an artifact, deployment installs it, and release exposes capability to users. Repeatable environments, compatibility checks, and recovery evidence connect these steps. Methodologies organize the people doing this work; they do not substitute for sound boundaries or feedback.
2.6 Tools: make intent and implementation inspectable
A useful toolchain covers the whole lifecycle: editors and code navigation for understanding; version control and review for change; compilers and dependency managers for reproducible builds; static analysis for fast feedback; tests and profilers for execution evidence; and pipelines, migration tools, logs, metrics, and traces for delivery and operation.
For architecture, dependency graphs, forbidden-reference rules, contract tests, and change-coupling analysis help compare intended boundaries with actual code. Complexity analysis can highlight difficult routines. Cyclomatic complexity counts independent control-flow paths; it does not measure domain difficulty, cross-module coupling, concurrency risk, or overall maintainability. Use it with context, change history, tests, and human judgment.
Architecture testing belongs in our CI/CD pipeline. Important boundaries should be executable checks: which modules may depend on each other, which layers may access persistence, and where cyclic dependencies are forbidden. A change that violates these rules should fail the pipeline before release. For Java projects, ArchUnit provides architecture tests through ordinary unit test frameworks. It checks package and class dependencies, layers, slices, and cycles by analysing Java bytecode. This makes architectural intent part of continuous verification, rather than a diagram we hope the code still follows.
Choose tools for signal quality, feedback speed, reproducibility, integration, ownership, privacy, maintenance cost, and the ability to leave the vendor. Keep configuration versioned, important versions pinned, and local and continuous-integration checks aligned. AI tools can draft, explain, generate, and review, but their output still needs a defined scope and evidence. Compilation, a coverage percentage, or a green pipeline alone does not establish correctness.
2.7 Putting the levels together
Consider reliable notifications through several providers. Separation of concerns and dependency inversion guide the decision. Strategy or Adapter may supply a useful pattern; interfaces, composition, and constructor injection implement it. Ports and adapters keep provider details outside application policy. A queue, retry policy, or circuit breaker may address a demonstrated remote-service failure. Refactoring improves the existing structure; delivery practices move the change safely; tests, dependency checks, and runtime telemetry provide evidence. Each concept has a role. Naming more patterns or buying more tools does not make the system easier to understand.
3. Processes
Compared with many physical artifacts, whose main structure can remain relatively stable after construction, software is a fast-evolving creature. It undergoes constant changes, from a small rule adjustment to a large redesign of its data, boundaries, or deployment. Our process therefore has to support both building the early versions and maintaining what follows. A plan drawn once and left behind cannot do that.
One useful starting point is the Domain-Driven Design Starter Modelling Process from DDD Crew. Its eight activities connect business understanding, domain discovery, boundaries, teams, and implementation. The authors describe an iterative learning guide, not a mandatory sequence or universal best practice.
Original overview by DDD Crew and contributors, reproduced unchanged. Source diagram · CC BY-SA 4.0 licence file · Open full-size diagram.
3.1 The eight activities
- Understand: establish the business goals, user needs, and outcomes the system should support.
- Discover: learn the domain collaboratively; techniques include EventStorming and Domain Storytelling.
- Decompose: identify cohesive subdomains and explore boundaries that reduce unnecessary coupling.
- Strategize: identify strategically important capabilities and decide where to invest effort.
- Connect: examine end-to-end interactions using message flows, process models, or sequence diagrams.
- Organise: align team responsibilities with context boundaries and practical organisational constraints.
- Define: clarify each bounded context's purpose, responsibilities, and significant design choices.
- Code: implement the domain model and use implementation feedback to refine the design.
The repository pairs these activities with tools such as the Business Model Canvas, Core Domain Charts, Bounded Context Canvas, and Aggregate Design Canvas. Revisit whichever activity new evidence demands.
3.2 Building and maintaining the same system
For an early version, I want enough planning to make the important commitments visible: the problem, the language we use, who owns each responsibility, and which boundaries the implementation must respect. That does not mean specifying every function before writing code. An experiment can expose a bad assumption faster than another diagram. The experiment should change our understanding deliberately, rather than quietly becoming the architecture by accident.
For an existing system, I want the same discipline applied to reality. Map what actually runs, who writes which data, where dependencies cross boundaries, and what changes together. Compare that picture with the intended design. When they disagree, decide whether to change the implementation, revise the design, or record a temporary exception. Then protect the important decision with a dependency rule, a contract test, a migration check, or operational evidence.
With AI agents, I would make those boundaries part of the task: what the agent may change, which contracts must hold, and what evidence it must return. Generated code, tests, and diagrams should agree with the running system. The process is doing its job when the next change requires less guesswork—not when we have produced more documentation.
4. Conclusion
Code is cheap. A system we can understand, verify, and keep changing is the real investment.



Conversation
Comments
Loading comments…