Skip to main content

Architecture Overview

info

AquaGen API follows a layered architecture pattern designed for scalability, maintainability, and testability. This document provides a comprehensive overview of the system architecture and its key components.

🏗️ High-Level Architecture​

🎯 Architectural Principles​

1. Separation of Concerns​

The application is organized into distinct layers, each with specific responsibilities:

  • Routes: HTTP request handling and response formatting
  • Services: Business logic and data orchestration
  • Formatters: Data transformation and presentation
  • Database: Data persistence and retrieval

2. API-First Approach​

RESTful API design with:

  • OpenAPI/Swagger documentation
  • Consistent request/response formats
  • Versioned endpoints for backward compatibility

3. Security by Design​

Enterprise-grade security:

  • JWT-based authentication
  • Azure AD integration
  • Input validation and sanitization
  • Secrets management with Key Vault

📦 Layered Architecture​

Layer Responsibilities​

1. Middleware Layer (app/__init__.py — auth_interceptor)​

  • Purpose: Intercept every inbound request before any route handler runs
  • Registered via: @app.before_request
  • Responsibilities:
    • Validate the JWT from the Authorization header
    • Decode the token and look up the full user + industry document
    • Populate current_user (industry metadata, shift time, daysOffset, permissions, nowUTCWithShiftDateTime) so all downstream route handlers and services can use it without repeating auth logic
    • Short-circuit with 401 if the token is missing, expired, or invalid (exempt routes like /login and /batchProcess bypass this check)
  • Admin-specific: Admin tokens carry industryId: "INTERNAL"; the middleware reads the targetIndustryId request header and loads the target industry's context into current_user for that request

2. Routes Layer (app/routes/)​

  • Purpose: Handle HTTP requests and responses
  • Files: 33 route files
  • Responsibilities:
    • Request parameter parsing
    • JWT token validation
    • Input validation
    • Response serialization
  • Example: report.py, device_data.py, alerts_routes.py

3. Services Layer (app/services/)​

  • Purpose: Implement business logic
  • Files: 40+ service classes
  • Responsibilities:
    • Data processing and aggregation
    • Business rule enforcement
    • Cross-functional orchestration
    • External API integration
  • Example: ReportService, AlertsProcessor, DeviceDataService

4. Formatters Layer (app/formatters/)​

  • Purpose: Transform data for presentation
  • Files: 52 formatter files
  • Responsibilities:
    • Data serialization
    • Template rendering
    • Format conversion (HTML → PDF, XLSX, CSV)
    • Response formatting
  • Example: ReportFormatter, DeviceDataFormatter

5. Database Layer (app/database/)​

  • Purpose: Abstraction layer between the database configuration and the rest of the project
  • Files: 3 core files
  • Responsibilities:
    • Initializes Cosmos DB client and container connections (database_config.py)
    • Exposes query methods used across services and routes (database_supporter.py)
    • Centralizes all Cosmos DB query strings (queires_list.py)
  • Key Classes: DatabaseSupporter (~200 methods)

🔄 Request Processing Flow​

Standard Request Flow​

🗃️ Database Architecture​

Cosmos DB Containers​

Container Purposes​

ContainerPurpose
industries_containerIndustry/facility metadata and configuration
users_containerUser accounts and profiles
devices_data_containerRaw IoT sensor readings
processed_data_containerAggregated and processed sensor data
notification_containerNotifications and push alert records
global_logs_containerSystem-wide audit and event logs
token_logs_containerJWT token issuance and revocation logs
industry_checkpoint_containerTracks industry-level edits made
standard_categories_master_containerLatest types of standard categories and their details
additional_feature_containerFeature flags and add-on configuration per industry
units_meta_containerMetadata for individual sensor units
alerts_config_containerAlert threshold and channel configuration
industry_metrics_containerLatest data snapshot for all units of an industry

🚀 Scalability Patterns​

1. In-Memory Caching​

  • CachedData: Standard categories and their mappings are loaded once at startup and reused across all requests, avoiding repeated database calls

2. Parallel Execution​

  • ThreadPoolExecutor: Used across database queries, device data fetching, water balance calculations, and formatter operations to run tasks concurrently

📊 Monitoring & Observability​

Telemetry Data​

  • Request Telemetry: HTTP requests, response times, and status codes — automatically captured via FlaskMiddleware and exported to Azure Application Insights

🔌 Integration Points​

1. IoT Integration​

  • Storm: Device-to-cloud messaging and telemetry ingestion for real-time sensor data
  • PLC via IoT Hub: Direct method calls to PLC devices for reading and writing register data

2. Notification Integration​

  • Firebase FCM: Push notifications to mobile apps
  • SMTP (Gmail): Email notifications and automated reports
  • SMS Gateway: OTP and alert text messages
  • Google Chat: Internal team notifications via webhook

3. External APIs​

  • PowerDrill: BI query platform integration
  • Google Earth Engine: Satellite-based water quality data for Lake Pulse

🎯 Design Patterns​

Architectural Best Practices

The patterns below are used consistently across the codebase to keep the system modular, testable, and easy to extend.

1. Service Locator Pattern​

  • CachedData: Global cache for frequently accessed data
  • DatabaseSupporter: Static methods for database access

2. Factory Pattern​

  • Report Type Factory: Dynamic report service instantiation
  • Formatter Factory: Format-specific formatter selection

3. Template Method Pattern​

  • Report Base Classes: Common report generation flow
  • Device Data Formatters: Polymorphic formatter templates

4. Decorator Pattern​

  • Authentication Decorators: @jwt_required()
  • Validation Decorators: @validate_values()
  • Admin Authorization: @admin_required()

📈 Performance Considerations​

1. Database Optimization​

warning

Cosmos DB queries must use industryId and date as partition keys wherever possible. Avoid cross-partition scans on high-volume containers like devices_data.

  • Partition Keys: Cosmos DB queries use industryId and date as partition keys for efficient data distribution
  • Selective Field Queries: Queries use explicit SELECT c.field projections instead of SELECT * to reduce payload size
  • Parallel Queries: ThreadPoolExecutor used for concurrent multi-unit and multi-date data fetching

2. Report Generation​

  • In-Memory File Streams: Reports are generated into BytesIO streams and returned directly without writing to disk
Architecture Best Practices

The AquaGen architecture follows cloud-native patterns and industry best practices for building scalable, maintainable applications.

📚 Next Steps​