Introduction
Astrocyte Data Gateway is engineered to optimize and secure data transfer between trading partners and cloud platforms, providing a comprehensive and adaptable solution.
This document presents the detailed architecture and deployment approach of the Astrocyte Data Gateway, aimed at guiding the setup and configuration of the data exchange system.
Architecture & Components
The application is structured into three integral components, each playing a crucial role in facilitating secure and effective data interchange.
Data Gateway UI
The Data Gateway UI is an essential interface designed for user-friendly management of various functionalities, including trading partner creation, user administration, file upload and download, file transfer search, access management, and cloud configurations. Through the UI, users can also manage cloud transfers and configure settings efficiently. The UI provides an intuitive and seamless way for users to interact with the system, streamlining operations and improving productivity by integrating directly with the underlying API.
Data Gateway API
The Data Gateway API is a crucial component responsible for handling various functionalities, including creating and managing trading partners, user management, file upload, download, file transfer search, access management, cloud configurations, creating and managing cloud transfers, and settings. This component exposes APIs for these operations, allowing seamless integration and interaction with other modules.
Data Gateway Server
Data Gateway Server serves as the heart of the product, providing APIs for protocol services using the Apache Mina library (SFTP, FTP, FTPS). It manages schedulers (polling interval), Watch Services, file locks, APIs to connect cloud storages (GCS, AWS S3, AZURE, IBM CLOUD), performs file transfers, handles secret manager services, utilizes event-based cloud triggers, manages logs, and determines the file transfer chunk size. This component is responsible for the core functionalities and secure data transmission between clouds and servers.
Data Gateway Secure
Data Gateway Secure plays a crucial role in enhancing security by restricting direct connections from trading partners to the server. It contains the same protocols implemented in the Data Gateway Server using the Apache Mina library. This component does not involve database connections. It acts as an intermediate layer to ensure secure and controlled access to the server.
System Requirements
The Data Gateway Application is compatible with the following system configurations:
Pre-Requisites
- Cloud Account (Google Cloud, Azure, AWS, IBM)
- Kubernetes Cluster
- Container images
- Helm packages
- Secret Manager
- Pub/Sub Service
- Persistent storage with access mode of RWO and RWX, and is exposed as a Kubernetes storage class
- Container Registry Access
- PostgreSQL Database
- SSL Certificates
- Data Gateway Image
- Linux Jump Server
- Elastic Search for Data Gateway Monitor (Optional)
