The Molecule Ecosystem

The Molecule ecosystem is the foundation of the Molecule Commodity Trading and Risk Management (CTRM) System.

Our approach is simple... invest in available cloud-based technologies to provide a rich and robust infrastructure while focusing our internal resources on the core application.

In this blog post, I'll outline which cloud-based technologies we leverage and on which parts of the ecosystem we focus our internal resources.

What We Leverage

Within the Molecule ecosystem, we use multiple third-party services and technologies to provide the reliability, performance, and user experience our customers have come to expect of us.

Infrastructure, Networking, & Backend Services

The majority of our system components run on top of Amazon Web Services (AWS) leveraging AWS' shared responsibility model.

At the core of our infrastructure is Kubernetes, managed by AWS EKS.

We rely on Kubernetes to make sure our actual system state matches our desired state.

This ensures system reliability and enables horizontal scalability to manage the intermittent system loads we need to support.

At Molecule, we run two Kubernetes clusters, one for our primary application and a second to host our microservices ("m.functions").

Along with EKS, we also leverage AWS Relational Database Services (RDS) to host and manage our PostgreSQL instance and AWS Elasticache to host and manage our Redis instances.

Hosting these critical infrastructure and back-end services on a cloud provider like AWS allows us to easily scale up and out as necessary so we can manage any load that our customers throw at us.

We can also easily execute version upgrades to keep up with advances in these technologies.

Supporting Services

In addition to the infrastructure and backend services managed by AWS, we use a variety of supporting services and technologies that fall into the following categories:

  • Business Intelligence
  • Security & Authentication Services
  • Logging & Monitoring

Business Intelligence

Instead of rolling out our own business intelligence and reporting solution, we leverage Mode Analytics, which allows Molecule to white label and embed the reports and data analytics we provide customers.

Mode provides the technology and data security and Molecule can quickly provide the reports our customers need.

Security & Authentication Services

Security of our platform and of our customers' data is Molecule's highest priority.

At Molecule, security begins with code development; however, we leverage multiple third-party services to provide a comprehensive security layer - from infrastructure security to application security.

Starting with the infrastructure layer, we leverage AWS networking and security services to protect the infrastructure within which the Molecule application runs.

We use VPCs (Virtual Private Clouds) to ensure the application and data layers do not provide any unnecessary access.

Weekly infrastructure scans are executed by AWS Inspector and AWS GuardDuty is integrated into our DevOps workflow to notify us immediately of any detected threats.

At all layers of the application, we use a growing set of tools to provide functions like single sign-on (SSO), penetration testing, and dynamic security scanning.

Logging & Monitoring

Managing application logs and monitoring of the application itself is another important area of the Molecule ecosystem where we use third-party services.

For managing application logs and providing access to log data, we use Elasticsearch and Kibana both hosted by

Molecule strives to be aware of any issues impacting our customers before our customers are and before our customers' business operations are impacted.

To help us achieve this objective, we use multiple monitoring and alerting services provided by Grafana, Pingdom, and Honeybadger.

Each of these services is integrated into our DevOps worfklow, which uses Slack as our primary DevOps user interface.

What We Build

To complement the areas where we rely on third-party services, we focus our internal resources on the following:

  • User Experience
  • Business Rules
  • Integrations
  • Microservices
  • Molecule API

User Experience

A primary directive within Molecule is that we do not compromise on user experience or usability of the application.

To achieve this level of user experience excellence, we have our own visual designer on staff who knows how to make software developers look good.

Business Rules

At Molecule, our engineers follow the clean architecture patterns from Robert (Uncle Bob) Martin.

At the heart of the architecture are enterprise and application business rules, which are the brain power of the Molecule application.

Molecule engineers spend a good portion of their time in this area of the ecosystem.


Another important objective at Molecule is to ease our users' lives by automating and streamlining as much of the trade capture and valuation process as we can.

To support this objective, we have developed a number of integrations with commodity exchanges (e.g. ICE and CME), ISOs (Independent Service Operators) through our relationship with Hartigen, market data providers (e.g. Morningstar), and FCMs (Futures Commission Merchant).

These integrations are a hybrid between third-party and internal focus since we use each provider's API.


In a prior post, Introducing Molecule M.Functions, I covered how Molecule uses Istio and Knative to provide an infrastructure for developing and deploying microservices.

These support the Molecule application and provide add-on capabilities that we develop on behalf of customers.

Microservices have allowed us to expand our development capabilities beyond our core engineers.

Molecule API

In the past couple of years, we have invested time building out and documenting a V2 REST API to expose all of Molecule's capabilities to our customers through an easily consumable API.

Our API has been used to develop custom integrations to applications hosted and managed by our customers as well as to pull trade and valuation data into Excel.

At Molecule, we eat our own dog food and our V2 API is also used by the Molecule application to support our V2 React screens.

If you are an existing customer, feel free to reach out to Molecule at to access the API docs and try the API for yourself.

Final Thoughts

Hopefully I've given you a glimpse into the rich ecosystem that makes up Molecule and into which areas of the Molecule CTRM system leverage cloud providers like AWS and third-party services and technology providers while internally we focus on our differentiators.

New Feature: Inventory Tickets

ETRM Build vs. Buy

New Feature: Custom Fields Galore


Introducing Molecule M.Functions​


Much has been said about micro-services over the last year and I wanted to find out for myself what the fuss was all about. At Molecule, we needed a way to augment our main application with supporting functions and services, and I figured it was time to give micro-services a shot. I looked at several open source technologies that promised to provide an easy-to-use micro-services platform. We ultimately landed on Knative given the strong community support around the technology and how the technology met our needs.

I used the following as guiding principles for evaluating the various micro-services technologies:

  1. Strong community support
  2. Integrates with our GitLab and Kubernetes Continuous Integration(CI)/Continuous Delivery(CD) tooling
  3. Supports our primary use cases
  4. Extends to our customers

Strong Community Support

As with any open source technology, strong community support is a must-have. Even if it's great tech, but does not have a good developer community supporting the technology, best to look for something else. In Knative's case, there is active community support on GitHub and a thriving Slack community. And of course, Google is behind the technology.​

Integrates With Our Existing CI/CD Tooling

It was important for me to not add yet another tool to our engineering workflow and to ensure we can leverage our existing CI/CD tooling. We were already heavy users of GitLab for our source code management and Docker image registry as well as AWS EKS (Kubernetes) to run the Molecule app. Our chosen micro-services platform needed to work within our current infrastructure.

To run our Knative instance, I ended up creating a separate AWS EKS cluster from our main production cluster to allow us to more easily upgrade our Knative cluster as needed and to avoid any unintended consequences to our production cluster. The following components are installed in the EKS Knative cluster:

  1. Knative Serving and Eventing
  2. Istio
  3. Kafka (as a messaging provider)

The Knative cluster consists of two primary namespaces, molecule-functions-dev for services under development and testing and molecule-functions that are serving production workloads. In GitLab, each function is a separate project with an accompanying .gitlab-ci.yml file to manage the build of the service and the auto-deployment to the appropriate namespace in the K8s cluster. When a change to a function is merged to the master code branch, the Knative service is built and auto-deployed to the molecule-functions-dev namespace. When a tag is applied to a function, the Knative service is auto-deployed to the molecule-functions namespace. Included in the build and deployment is the K8s service.yaml file which describes the service and any supporting deployments such as a cronjob.yaml, if one is required for scheduled services.

With this approach, our developers as well as our customer success team can live in their favorite IDE, and easily develop, maintain, and deploy scalable Knative services simply by committing and tagging code in GitLab.​

Supports Our Primary Use-Cases

At Molecule, our chosen micro-services platform needed to support the following use-cases:

  1. Can it run on a schedule?
  2. Can it be executed on-demand?
  3. Does it support messaging?
  4. Can it scale?

On a Schedule

We have multiple instances where we need to run a function on a schedule to allow our customer success team or our development team to execute routine functions that allow us to maintain the health of the Molecule app. Knative's CronJobSource provides a straight forward mechanism to schedule these routine functions. Examples of routine scheduled functions include data integrity checks, cleaning up obsolete log data, and nightly resetting our alerting system.


To help extend the core Molecule application, our services needed to be reachable from the Molecule app that would allow us to execute custom services that we develop on behalf of our customers or services that we want to run outside of our production cluster. For example, our asset valuation module can call external services either hosted by our customers or hosted by us that returns custom valuation data to Molecule. These services can now run in our Knative cluster allowing us to scale these services as needed.

Message Based

Guaranteed and reliable messaging is critical to any enterprise system, particularly commodity trading and risk management systems like Molecule. We reliably utilize messaging today within our core Molecule app to scale our back-end processes; however, we also have a need to utilize messaging outside of our core app to support business critical functions on behalf our customers. For example, our batch reporting functionality relies on messaging to distribute the creation and publication of batch reports (e.g. trade confirms) to our customers. In our Knative cluster, we utilize Kafka as a guaranteed message transport to ensure a message reaches its destination and performs the intended function.


Scalability is another critical component of a good micro-services architecture that supports the scaling of services to meet demand as well as scaling to zero for non-critical services that can tolerate a cold start. Knative supports this requirements nicely by allowing us to set scaling parameters at a per service level supporting scaling to 0, setting a default number of always-on service replicas to respond to normal demand, and setting max scaling levels to control the workload in peek demand.​

Extends to Our Customers

A primary reason for adding a micro-services based approach to our ecosystem is to better support our customers and to provide them with the functionality that is required to do their business without having to resort to core application changes. As stated above, our Knative cluster allows us to quickly and easily develop, deploy, and scale business critical services to augment our core application and also enables us to better serve our customers. This platform also positions us to provide future services to our customers such as our customers running their own developed and managed services in our managed Knative cluster to further extend their use of Molecule.


With all the benefits and capabilities of a micro-services platform outlined above, we have embraced micro-services as an extension to our platform and we are now referring to this collected set of capabilities as Molecule's m.functions.

New Feature: Inventory Tickets

Release 100 3/4

Djinn - Enterprise Crypto by Molecule