The Architecture Behind the Magic

"Any fool can write code that a computer can understand. Good programmers write code that humans can understand." — Martin Fowler

When I first started building our connector platform at Youkti, I thought design patterns were academic concepts — things you read about in textbooks but rarely use in real projects. I was wrong. Dead wrong. As our platform grew from handling a handful of OAuth integrations to managing enterprise-scale connector ecosystems, I discovered that design patterns aren't just theoretical — they're survival tools.

This blog is the story of how we went from spaghetti code to a well-architected system, one pattern at a time. More importantly, it's about why each pattern matters and how they work together to create something greater than the sum of their parts.

The Problem: When Good Intentions Meet Reality

Our connector platform started like most projects — with urgency. We needed to integrate with Gmail, so we wrote a Gmail service. Then Outlook, so we wrote an Outlook service. Then Salesforce, Teams, Slack, and suddenly we had six different services with six different approaches to the same problems: authentication, error handling, logging, and tenant isolation.

The codebase was a minefield. Change something in one service, and you'd have to remember to change it in five others. Authentication logic was duplicated everywhere. Error handling was inconsistent. Testing was nearly impossible because everything was tightly coupled.

That's when I realized: we didn't have a connector problem. We had an architecture problem. And architecture problems are solved with design patterns.

The Patterns: Building Blocks of Enterprise Software

Over the following months, we refactored our entire platform around twelve core design patterns. Each pattern solved a specific problem, and together they created a system that's maintainable, testable, and scalable. Let me walk you through each one.

1. Service Layer Pattern

The Problem: Our API routes were doing everything — handling HTTP requests, validating input, executing business logic, talking to databases, and formatting responses. A single endpoint could be 200 lines of tangled code.

The Solution: We extracted all business logic into dedicated service classes. Now our API routes are thin — they handle HTTP concerns and delegate everything else to services like ConnectorService, TeamsService, and SalesforceService.

How It Works: Each service class encapsulates all operations for a specific domain. The ConnectorService handles creating, listing, updating, and deleting connectors. It doesn't know anything about HTTP status codes or request parsing — that's the API layer's job.

Benefits:

• ●Separation of Concerns: API routes handle HTTP, services handle business logic
• ●Testability: Services can be unit tested without spinning up an HTTP server
• ●Reusability: The same service can be used by multiple API endpoints
• ●Maintainability: Changes to business logic don't affect API contracts

2. Dependency Injection Pattern

The Problem: Our services were creating their own dependencies. A ConnectorService would instantiate its own database connection, its own cache client, its own encryption manager. Testing was a nightmare because you couldn't swap out dependencies.

The Solution: We adopted FastAPI's Depends() system religiously. Every dependency — database sessions, current user, current tenant, rate limiters — is injected from the outside.

How It Works: When an API endpoint is called, FastAPI automatically resolves and injects all dependencies. The endpoint declares what it needs (like "I need a database session and the current user"), and the framework provides them. Components never create their own dependencies.

Benefits:

• ●Loose Coupling: Components don't create their own dependencies
• ●Testability: Easy to mock dependencies in tests
• ●Flexibility: Can swap implementations without changing consumers
• ●Automatic Lifecycle: FastAPI manages dependency lifecycle automatically

3. Factory Pattern

The Problem: Object creation was scattered throughout the codebase. Every time we needed a TeamsService, we'd write out the full initialization. If the constructor changed, we'd have to update every call site.

The Solution: We introduced factory functions like create_teams_service() and get_audit_logger(). All object creation logic is centralized in one place.

How It Works: Instead of calling constructors directly, code calls factory functions. The factory knows how to properly initialize the object, what dependencies to inject, and can even implement lazy initialization or caching. Our cache system uses a factory pattern — cache_get_or_set() takes a factory function that's only called if the value isn't cached.

Benefits:

• ●Encapsulation: Object creation logic is centralized
• ●Flexibility: Easy to change how objects are created
• ●Consistency: All instances created the same way
• ●Lazy Initialization: Objects created only when needed

4. Singleton Pattern

The Problem: Some resources should only exist once — like our Redis cache connection or encryption manager. Creating multiple instances wastes memory and can cause inconsistencies.

The Solution: We created global singleton instances for shared resources: cache_manager, encryption_manager, tenant_context, and token_cache. These are instantiated once at startup and imported wherever needed.

How It Works: Each singleton is created as a module-level variable. When any part of the application imports it, they get the same instance. The cache_manager maintains a single connection pool to Redis that's shared across all requests.

Benefits:

• ●Resource Efficiency: Single connection pool shared across requests
• ●State Consistency: All code uses the same instance
• ●Memory Efficiency: No duplicate instances
• ●Easy Access: Import and use anywhere

5. Context Manager Pattern (Async)

The Problem: Every operation needed the same boilerplate: start a timer, check rate limits, execute the operation, log success or failure, handle cleanup. This code was duplicated everywhere, and inevitably some operations would forget error handling or logging.

The Solution: We created async context managers like _operation_context() and audit_context() that wrap operations with automatic timing, logging, and error handling.

How It Works: When you wrap code in an async context manager, it automatically executes setup code before your operation (like checking rate limits and starting a timer), yields control to your code, then executes cleanup code afterward (like logging the result and duration). If an exception occurs, it's caught, logged, and re-raised.

Benefits:

• ●Automatic Cleanup: Resources released even on exceptions
• ●Consistent Timing: All operations automatically timed
• ●Automatic Logging: Success/failure logged without boilerplate
• ●DRY Principle: No repeated try/finally blocks

6. Repository Pattern

The Problem: Database queries were embedded directly in business logic. If we wanted to change from PostgreSQL to another database, we'd have to rewrite everything. Worse, tenant isolation filters were sometimes forgotten.

The Solution: We implemented repository-style methods in our services, along with a TenantQueryFilter helper that automatically applies tenant isolation to all queries.

How It Works: All database access goes through repository methods like list_tenant_connectors() and get_connector_by_id(). These methods always apply tenant filtering, handle pagination, and return domain objects. The TenantQueryFilter.apply_tenant_filter() helper makes it impossible to forget tenant isolation.

Benefits:

• ●Abstraction: Business logic doesn't know about SQL
• ●Tenant Isolation: All queries automatically filtered by tenant
• ●Reusability: Common query patterns centralized
• ●Testability: Can mock repository for tests

7. Circuit Breaker Pattern

The Problem: When external services like Composio went down, our entire platform would slow to a crawl. Every request would wait for timeout, hammering a service that was already struggling.

The Solution: We implemented a circuit breaker in our ComposioManager. After a certain number of failures, the circuit "opens" and subsequent requests fail immediately without even trying.

How It Works: The circuit breaker tracks success and failure counts. When failures exceed a threshold (say, 5 in a row), it opens the circuit. For the next 60 seconds, all requests fail immediately. After the recovery timeout, it allows one request through — if it succeeds, the circuit closes and normal operation resumes.

Benefits:

• ●Fault Tolerance: Prevents cascading failures
• ●Fast Failure: Fails fast when service is down
• ●Self-Healing: Automatically recovers after timeout
• ●Resource Protection: Stops hammering failing services

8. Decorator Pattern

The Problem: Cross-cutting concerns like authorization, tenant scoping, and role checking were being duplicated in every endpoint. Adding a new security check meant modifying dozens of functions.

The Solution: We created decorators like @tenant_scoped, @require_roles(), and @require_tenant_access() that can be applied to any function to add these concerns declaratively.

How It Works: A decorator wraps a function with additional behavior. When you apply @require_roles("admin") to an endpoint, the decorator checks the user's roles before the endpoint code runs. If the check fails, it raises an exception. The endpoint code itself doesn't need to know about authorization.

Benefits:

• ●Cross-Cutting Concerns: Security and logging applied declaratively
• ●DRY Principle: No repeated authorization code
• ●Readability: Intent clear from decorator name
• ●Composability: Multiple decorators can be stacked

9. Strategy Pattern

The Problem: Different connector types needed different behaviors — different OAuth scopes, different API endpoints, different data formats. We had massive if-else chains checking connector types.

The Solution: We implemented the strategy pattern through configuration dictionaries and conditional encryption. Each connector type maps to its own set of scopes and behaviors.

How It Works: Instead of if-else chains, we use dictionaries that map connector types to their configurations. When we need the scopes for Outlook, we look up CONNECTOR_DEFAULT_SCOPES["outlook"]. Adding a new connector type means adding a new entry to the dictionary, not modifying existing code.

Benefits:

• ●Flexibility: Easy to add new connector types
• ●Open/Closed Principle: Open for extension, closed for modification
• ●Runtime Selection: Strategy chosen based on connector type
• ●Testability: Each strategy tested independently

10. Middleware Pattern

The Problem: Every request needed authentication and tenant context setup. We were duplicating this logic at the start of every endpoint.

The Solution: We implemented ASGI middleware classes — AuthMiddleware and TenantMiddleware — that process every request before it reaches any endpoint.

How It Works: Middleware sits between the web server and your application. When a request comes in, it passes through AuthMiddleware (which validates the JWT), then TenantMiddleware (which sets up tenant context), before finally reaching your endpoint. The middleware can also run cleanup code after the response is sent.

Benefits:

• ●Cross-Cutting: Applied to all requests automatically
• ●Separation: Auth/tenant logic separate from business logic
• ●Chainable: Multiple middleware in sequence
• ●Transparent: Routes don't know about middleware

11. Builder Pattern

The Problem: Building database queries with optional parameters was messy. We had complex conditionals to add filters, sorting, and pagination.

The Solution: We leveraged SQLAlchemy's fluent query interface as a builder pattern, constructing queries incrementally by chaining method calls.

How It Works: You start with a base query, then chain on additional conditions. Each method returns the modified query, so you can keep chaining. If a filter parameter is None, you simply don't add that condition. The query is only executed when you call execute().

Benefits:

• ●Fluent Interface: Readable, chainable method calls
• ●Flexibility: Build complex queries incrementally
• ●Optional Parameters: Easy to add/remove conditions
• ●Deferred Execution: Query built before execution

12. Observer Pattern (via Audit Logging)

The Problem: Writing audit logs synchronously to the database slowed down every operation. But we couldn't lose audit data, especially for failures and security events.

The Solution: We implemented an AuditLogger with buffered logging. Events are collected in a buffer and flushed periodically, but critical events trigger immediate flushes.

How It Works: The audit logger maintains an internal buffer of log entries. Every 10 seconds, a background task flushes the buffer to the database. But if a failure or security event occurs, or if the buffer gets too full, it flushes immediately. This gives us the best of both worlds: performance for normal operations and reliability for critical events.

Benefits:

• ●Decoupling: Audit logging doesn't block main flow
• ●Batching: Efficient bulk database writes
• ●Reliability: Critical events flush immediately
• ●Performance: Normal operations aren't slowed down

How It All Fits Together

These twelve patterns don't exist in isolation — they work together in a layered architecture that separates concerns and maintains clean boundaries.

The API Layer (FastAPI routers) handles HTTP concerns and uses Dependency Injection to get what it needs. The Middleware layer processes every request for authentication and tenant context.

The Service Layer contains all business logic. Services use Context Managers for automatic timing and logging, Repository methods for data access, and Strategy patterns for connector-specific behavior.

The Core Layer provides shared infrastructure: the ComposioManager with its Circuit Breaker, the EncryptionManager and CacheManager as Singletons, and Factory functions for object creation.

The Data Layer handles persistence to PostgreSQL, caching in Redis, and external API calls to Composio.

What I Learned Along the Way

Patterns solve problems, not requirements. Don't implement a pattern because you read about it — implement it because you have a specific problem it solves. We added the circuit breaker only after external service failures started causing cascading problems.

Patterns compose beautifully. The real magic happens when patterns work together. Dependency Injection makes the Service Layer testable. Context Managers make the Repository Pattern consistent. Middleware and Decorators both address cross-cutting concerns at different levels.

Async changes everything. Classic pattern implementations assume synchronous code. We had to adapt every pattern for Python's async/await world — async context managers, async factory functions, async middleware. The patterns still work, but the implementation details matter.

Security must be baked in. Every pattern in our architecture includes tenant isolation. The Repository Pattern filters by tenant. The Middleware Pattern sets tenant context. The Context Manager Pattern logs tenant IDs. Security isn't a feature — it's a property of the architecture.

Patterns make onboarding easier. When new engineers join the team, they can understand the codebase because it follows recognizable patterns. "Oh, this is a Service Layer" is a lot more helpful than "Oh, this is... something."

The Journey Continues

Design patterns aren't a destination — they're a journey. Our architecture continues to evolve as we encounter new problems and discover new solutions. But the foundation of these twelve patterns has proven remarkably stable.

If you're building a connector platform — or any enterprise software — I hope this exploration of our patterns gives you ideas for your own architecture. The specific implementations matter less than the principles: separate concerns, inject dependencies, centralize object creation, and make the right thing easy and the wrong thing hard.

Your future self — debugging a production issue at 2 AM — will thank you for taking the time to get the architecture right.