Nginx (web server)
Nginx is a popular open source light weight modular web server. Nginx server is responsible for handling static resources and act as proxy to the internal service. It logs the access, errors with actual user ip address, request URIs and response times. It can forward all media requests to Media server, set caching policies, define security headers etc.
Databases (Maria DB)
MariaDB is community-driven fork of the MySQL relational database management system intended to remain free under GNU-GPL. The following is the list of separate instances of database resources which can be deployed as different schemas as well.
- Logistics/Master DB - will contain all data related to Logistimo application like Users, Entities, Transactions, Orders, Inventory etc.
- Assets DB - this database will exclusively be used by the AMS which managed the Assets, Temperature data, Alarms etc.
- Archivals DB - Historical data from Logistics and Assets DB is purged to this database once in a month, based on the archival policy defined. This is necessary to retain the performance of the database machines, as data tends to grow significantly over time. For e.g. Raw temperature data is kept only for a duration of 12 months in the primary Assets DB.
Active MQ
Active MQ is an open source Messaging queue implementation. Task service utilises this to enqueue and distribute tasks to the task servers. This MQ provides persistence capability ensuring that no task is lost. It also provides failover capability using Zookeeper and levelDB implementation.
Zookeeper
ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them ,which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.
Zookeeper is used by the ActiveMQ to do cluster co-ordination, elect master. Cron services in Logistics and AMS also requires leader election, so that only one server is scheduling tasks at specified times.
Cache (Redis)
Redis is an extremely fast in memory key value database that supports many fundamental data structures like Lists, Sorted lists, Sets, Sorted sets etc. It also through Redis sentinels supports failover capabilities in near real time.
Media server (Hadoop)
Hadoop DFS is a scalable distributed file system. This DFS will hold media objects like images, and also for storing exports, custom report templates and reports. Hadoop is a append only file system which makes it an excellent candidate for doing large volume exports by partially writing data as it is available. It also internally manages replication of the data, so data is never lost in the cluster. This can be controlled by the replication setting in configuration. It also curbs the security risks of the files uploaded by users, as the physical file is never stored on the file system in it’s intended format.