Network architecture
Network architecture overview
HeartAI deploys to virtual networks with implementations that are provided by Microsoft Azure. Network instances typically deploy through a hub-spoke network topology with the hub instance implemented with Azure Virtual WAN (vWAN) with connectivity through virtual network peering between the virtual network and Azure vWAN hub instances. This allows HeartAI to deploy into existing organisational network topologies that implement an Azure vWAN hub with a well-defined connectivity mechanism. HeartAI network implementations are often private networks, and with the additional application of virtual network traffic routing this enforces network traffic to route through the corresponding organisational network topology to ensure that appropriate logging, monitoring, and access control is achieved.
HeartAI deploys supporting functionalities including Azure Private DNS to provide internal name resolution, Azure Network Watcher for logging and monitoring of network instances, and Azure Network security groups for network traffic control. HeartAI services do not expose network endpoints to the public internet - Service network resolution occurs internal to the HeartAI network or through private network extension.
HeartAI instances of Red Hat OpenShift provide native support for network management through first-class software-defined networking (SDN). This allows the OpenShift environment to operate as a virtual private cloud with OpenShift namespaces that are effectively separated from each other by tunnelled overlay network VXLAN. These implementations are natively provided by OpenShift integration with Open vSwitch. OpenShift allows location transparent name resolution to occur across a distributed cluster with name resolution provided with integrated CoreDNS instances. HeartAI typically deploys OpenShift through Microsoft Azure Red Hat OpenShift, such that OpenShift network instances are themselves virtualised as overlay networks on top of Azure virtual network instances.
HeartAI services are accessible through OpenShift service endpoint routes that are resolvable within the HeartAI domain and through the domains of extending networks. These endpoint routes provide network proxy to corresponding load-balanced reverse-proxies that are implemented with OpenShift services.
In addition to the SDN capabilities provided by OpenShift, HeartAI networking also implements OpenShift Service Mesh, providing advanced mechanisms for communication across services. The cloud-native service-mesh software Istio extends the OpenShift SDN with programmable and application-aware declarative network implementations. A core feature of Istio is the Envoy service proxy that is injectable as a sidecar into virtual IP hosts of the OpenShift SDN. Istio provides general approaches for network deployments, routing, traffic management, telemetry, and security. The management console Kiali provides capabilities for configuration, eventing, metrics, visualisation, and validation of network deployments that are implemented with Istio. Kiali allows for the display of service mesh structure by inferring traffic topology and health status. Kiali also provides native integration for the Grafana observability platform, and the Jaeger distributed tracing software.
Encryption over TLS occurs wherever possible for internal and external network communication. For example, service instances may communicate with each other via TCP over TLS, HTTPS, or WebSockets Secure. Similarly, backing services, such as persistent data stores, communicate with service instances through these mechanisms. Encryption also occurs for communication within OpenShift environments between the system control plane and compute nodes as well as between system components. The combined functionality of these approaches allows HeartAI network instances to be implemented as zero trust networks. The HeartAI OpenShift environment provides automated approaches for certificate issuance and management. This is primarily managed with the certificate management software cert-manager. cert-manager automates the issuance of certificates by utilising the ACME protocol, which also supports certificate rotation and revocation.
Further information about the HeartAI domain name architecture may be found with the following documentation:
Further information about the HeartAI system infrastructure may be found with the following documentation:
Azure network architecture
Azure Virtual Network provides cloud-hosted networking infrastructure. HeartAI services do not expose network endpoints to the public internet - All network resolution occurs internal to the HeartAI network or through private network extension.
The following example shows a typical HeartAI Azure network implementation with two Azure Virtual Network instances. The first network is specific for a HeartAI production environment instance while the second network is specific for a HeartAI testing environment instance.
HeartAI Azure Virtual Network address spaces
Network name | Description | Network address range | Hosting network |
---|---|---|---|
sah-heartai-vnet-prod-aue-001 | HeartAI production environment | 10.X.X.0/24 | Microsoft Azure |
sah-heartai-vnet-test-aue-001 | HeartAI testing environment | 10.X.X.0/24 | Microsoft Azure |
The following example shows the partitioning of a HeartAI production environment Azure Virtual Network instance to the following subnetworks:
HeartAI production environment subnetworks
Subnet name | Description | Subnet address | Address range | Available IPs | Hosts |
---|---|---|---|---|---|
sah-heartai-snet-aroworker-prod-aue-001 | Azure Red Hat Openshift worker nodes | 10.X.X.0/25 | 10.X.X.0 - 10.X.X.127 | 10.X.X.1 - 10.X.X.126 | 126 |
sah-heartai-snet-aromaster-prod-aue-002 | Azure Red Hat Openshift master nodes | 10.X.X.128/27 | 10.X.X.128 - 10.X.X.159 | 10.X.X.129 - 10.X.X.158 | 30 |
Subnet range unassigned | 10.X.X.160/27 | 10.X.X.160 - 10.X.X.191 | 10.X.X.161 - 10.X.X.190 | 30 | |
sah-heartai-snet-paas-prod-aue-004 | Azure PaaS endpoints | 10.X.X.192/26 | 10.X.X.192 - 10.X.X.255 | 10.X.X.193 - 10.X.X.254 | 62 |
Terraform declaration
The HeartAI implementation of Microsoft Azure is managed with the Terraform declarative infrastructure-as-code software framework. Terraform allows for the declaration of system components using configuration files specified with the HashiCorp Configuration Language (HCL). Further information about the HeartAI implementation of Terraform may be found with the following documentation:
The following example shows Terraform declarations to specify an Azure Virtual Network. This implementation coordinates instances of the following Azure components:
Azure component | Functionality for Azure Virtual Network |
---|---|
Private DNS | Provides private name resolution for Azure Storage Account network endpoints within the corresponding Azure Private DNS zone |
Virtual Network | Configures and deploys an Azure Virtual Network |
Network Watcher | Integrates an instance of Azure Network Watcher to a corresponding Azure Virtual Network |
resource "azurerm_private_dns_zone" "heartai-dns-private" {
name = "sah.heartai.net"
resource_group_name = azurerm_resource_group.rg.name
tags = {
"Application Name" = var.heartai-environment.application-name
"Application Owner" = var.heartai-environment.application-owner
"Environment" = var.heartai-environment.environment
"Division Department" = var.heartai-environment.division-department
"Cost Centre" = var.heartai-environment.cost-centre
}
}
resource "azurerm_virtual_network" "vnet" {
name = "sah-heartai-vnet-prod-aue-001"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
address_space = [
var.vnet-prod-aue-001-address-space.vnet]
tags = {
"Application Name" = var.heartai-environment.application-name
"Application Owner" = var.heartai-environment.application-owner
"Environment" = var.heartai-environment.environment
"Division Department" = var.heartai-environment.division-department
"Cost Centre" = var.heartai-environment.cost-centre
}
}
resource "azurerm_subnet" "snet_aroworker" {
name = "sah-heartai-snet-aroworker-prod-aue-001"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefixes = [
var.vnet-prod-aue-001-address-space.snet1]
service_endpoints = [
"Microsoft.ContainerRegistry"]
enforce_private_link_endpoint_network_policies = true
}
resource "azurerm_subnet" "snet_aromaster" {
name = "sah-heartai-snet-aromaster-prod-aue-002"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefixes = [
var.vnet-prod-aue-001-address-space.snet2]
service_endpoints = [
"Microsoft.ContainerRegistry"]
enforce_private_link_service_network_policies = true
}
resource "azurerm_subnet" "snet_vpn_gateway" {
name = "GatewaySubnet"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefixes = [
var.vnet-prod-aue-001-address-space.snet3]
enforce_private_link_endpoint_network_policies = true
}
resource "azurerm_subnet" "snet_paas" {
name = "sah-heartai-snet-paas-prod-aue-004"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefixes = [
var.vnet-prod-aue-001-address-space.snet4]
enforce_private_link_endpoint_network_policies = true
}
resource "azurerm_network_watcher" "network_watcher" {
name = "NetworkWatcher_australiaeast"
location = "australiaeast"
resource_group_name = "NetworkWatcherRG"
tags = {
"Application Name" = var.heartai-environment.application-name
"Application Owner" = var.heartai-environment.application-owner
"Environment" = var.heartai-environment.environment
"Division Department" = var.heartai-environment.division-department
"Cost Centre" = var.heartai-environment.cost-centre
}
}
Azure deployment overview
The following figure shows a structural overview of Microsoft Azure cloud resources within a HeartAI production environment instance. This figures represents:
- A corresponding Azure vWAN hub, including:
- An Azure ExpressRoute as an example of external network connectivity.
- An Azure Virtual WAN instance.
- Network peering between a HeartAI Azure Virtual Network instance and a corresponding Azure vWAN hub.
- A HeartAI Azure Virtual Network instance, with the following contained resources:
- Azure Red Hat OpenShift Master nodes.
- Azure Red Hat OpenShift Worker nodes.
- Azure private endpoints, with internal and private network connectivity to Azure cloud services.
- Azure cloud services, including:
Approaches for access to organisational networks
HeartAI end-users, developers, and administrators may access on-premises network instances and Azure tenancies in coordination with a hosting organisation. Access typically occurs through the three following approaches:
Azure network access approach | Description | Network access type |
---|---|---|
Organisation foreign network point-to-site VPN | Point-to-site network access from an end-users gateway to the foreign network VPN gateway of a hosting organisation. | On-premises point-to-site VPN |
Azure Virtual WAN point-to-site VPN | Point-to-site network access from an end-users gateway to an Azure VPN Gateway within an Azure Virtual WAN instance of a hosting organisation. | Cloud point-to-site VPN |
Azure Virtual Network point-to-site VPN | Point-to-site network access from an end-users gateway to an Azure VPN Gateway within an Azure Virtual Network instance of a hosting organisation. | Cloud point-to-site VPN |
Example: Organisation foreign network point-to-site VPN
The following figure shows an example organisation foreign network point-to-site VPN, representing access from an end-user device to the foreign network gateway of a hosting organisation:
Example: Azure Virtual WAN point-to-site VPN
The following figure shows an example Azure Virtual WAN point-to-site VPN, representing access from an end-user device to the Azure VPN Gateway within an Azure Virtual WAN instance of a hosting organisation:
Example: Azure Virtual Network point-to-site VPN
The following figure shows an example Azure Virtual Network point-to-site VPN, representing access from an end-user device to the Azure VPN Gateway within an Azure Virtual Network instance of a hosting organisation:
Organisational DNS zone architecture
For deployments within hosting organisational networks, HeartAI typically requests to manage a private DNZ zone. In the most direct DNS zone architecture deployment, the following DNZ zones are defined:
DNZ zone | Description | Managing organisation | Corresponding network |
---|---|---|---|
[hostorg].[tld] | An organisational DNS zone. | Hosting organisation | An organisational network. |
[hostorg].heartai.net | An instance of Azure Private DNS zone. | HeartAI | A private Azure Virtual Network as a private extension of the organisation network. |
Further information about HeartAI domain name architecture may be found with the following:
The following example shows a typical DNS zone architecture within a hosting organisational network structure. The corresponding organisation and HeartAI DNS zones are represented with the network constructs between these environments.
HeartAI private DNS orchestration within a deployed environment is also important for the automation of cryptographic certificates.
Within HeartAI OpenShift environment provides automated approaches for certificate issuance and management. This is primarily coordinated with the certificate management software cert-manager. cert-manager automates the issuance of certificates by utilising the ACME protocol, which also supports certificate rotation and revocation. The corresponding certificate authority is Let’s Encrypt. The preferred challenge mechanism to prove HeartAI domain ownership is with the DNS01 challenge. cert-manager performs this challenge by communicating with the HeartAI global domain name servers that are implemented with Cloudflare DNS. This allows HeartAI domain hostnames to issue trustable certificates.
Further information about HeartAI network security may be found with the following:
OpenShift network architecture
HeartAI instances of Red Hat OpenShift provide native support for network management through first-class software-defined networking (SDN). This allows the OpenShift environment to operate as a virtual private cloud with OpenShift namespaces that are effectively separated from each other by tunnelled overlay network VXLAN. These implementations are natively provided by OpenShift integration with Open vSwitch. OpenShift allows location transparent name resolution to occur across a distributed cluster with name resolution provided with integrated CoreDNS instances. HeartAI typically deploys OpenShift through Microsoft Azure Red Hat OpenShift, such that OpenShift network instances are themselves virtualised as overlay networks on top of Azure virtual network instances.
OpenShift service endpoint architecture
HeartAI services are accessible through OpenShift service endpoint routes that are resolvable within the HeartAI domain and through the domains of extending networks. These endpoint routes provide network proxy to corresponding load-balanced reverse-proxies that are implemented with OpenShift services.
OpenShift client endpoint architecture
System end-users may also access the system through clients that act on behalf of the end-user. Network access to these clients is similar to the direct services access approaches described above. System protected resources are accessible to these clients provided that the end-user has delegated the corresponding authority to the client for these purposes.
Please refer to the documentation section Identity and access management for an overview of these approaches.
OpenShift proxied client endpoint architecture
In addition to client network resolution, clients may be resolved through sidecar network reverse-proxy. The primary implementation with this approach is through OAuth 2 Proxy. This approach is particularly useful for legacy clients that have a limited ability to independently perform the OAuth 2.0 Authorization Code Grant.
Please refer to the documentation section Identity and access management for an overview of these approaches.
OpenShift Service Mesh implementation
In addition to the SDN capabilities provided by OpenShift, HeartAI networking also implements OpenShift Service Mesh, providing advanced mechanisms for communication across services. The cloud-native service-mesh software Istio extends the OpenShift SDN with programmable and application-aware declarative network implementations. A core feature of Istio is the Envoy service proxy that is injectable as a sidecar into virtual IP hosts of the OpenShift SDN. Istio provides general approaches for network deployments, routing, traffic management, telemetry, and security. The management console Kiali provides capabilities for configuration, eventing, metrics, visualisation, and validation of network deployments that are implemented with Istio. Kiali allows for the display of service mesh structure by inferring traffic topology and health status. Kiali also provides native integration for the Grafana observability platform, and the Jaeger distributed tracing software.
OpenShift Service Mesh composes the following technologies:
OpenShift Service Mesh technology | Description | Reference |
---|---|---|
Istio | Modern service mesh that is natively implemented with Kubernetes networking capabilities. Supports advanced network routing and monitoring through the use of injectable sidecar proxies. | https://istio.io/ |
Kiali | Management console for observability and management of Istio implementations. | https://kiali.io/ |
Jaeger | End-to-end distributed tracing. First-class integration with Istio and Kiali. | https://www.jaegertracing.io/ |
Istio implementation
The cloud-native service-mesh software Istio extends the OpenShift SDN with programmable and application-aware declarative network implementations. A core feature of Istio is the Envoy service proxy that is injectable as a sidecar into virtual IP hosts of the OpenShift SDN. Istio provides general approaches for network deployments, routing, traffic management, telemetry, and security.
Further information about the HeartAI Istio implementation may be found with the following documentation:
Example implementation components
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: heartai-hello-gw
namespace: heartai-hello-world-prod
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
tls:
httpsRedirect: true
hosts:
- hello.prod.apps.aro.sah.heartai.net
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: heartai-hello-world-prod-gw-cert
hosts:
- hello.prod.apps.aro.sah.heartai.net
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: heartai-service-hello-world-vs-prod
namespace: heartai-hello-world-prod
spec:
hosts:
- heartai-service-hello-world-prod.heartai-hello-world-prod.svc.cluster.local
gateways:
- mesh
# - heartai-service-hello-world-gw
http:
- match:
- uri:
prefix: "/"
route:
- destination:
host: heartai-service-hello-world-prod
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: heartai-service-hello-world-rule-prod
namespace: heartai-hello-world-prod
spec:
host: heartai-service-hello-world-prod
subsets:
- name: stable
labels:
version: v0.33.44
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: postgresql
namespace: heartai-hello-world-prod
spec:
hosts:
- sah-heartai-psql-prod-aue-001.postgres.database.azure.com
ports:
- number: 5432
name: postgresql
protocol: tcp
resolution: DNS
location: MESH_EXTERNAL
Kiali implementation
Kiali provides capabilities for configuration, eventing, metrics, visualisation, and validation of Istio service mesh software. Kiali allows for the display of service mesh structure by inferring traffic topology and health status, and supports reporting and visualisation with:
- Graphs displaying service mesh topology and real-time traffic flow.
- Service mesh status and health updates.
- An overview of service mesh components, including associated workloads and services.
- Reports and status checks for inbound and outbound traffic.
- Metrics and visualisation for inbound and outbound traffic.
- Tooling for distributed service mesh tracing.
HeartAI instances of Red Hat OpenShift implementation are extendable with OpenShift Service Mesh, integrating Istio, Kiali, and the Jaeger distributed tracing software.
Further information about the HeartAI Kiali implementation may be found with the following documentation:
The following image shows an example of the Kiali management console with an implementation of Istio for the heartai-hello-world-dev
namespace. The components of this namespace relate to the HeartAI HelloWorldService
instance and show the service gateway and reverse-proxy, and the backing services PostgreSQL (postgresql
) and Apache Kafka (kafka-brokers
):
The following images are from the HeartAI interface with the SA Health Health Information Broker, a major provider of HL7 formatted health data within SA Health:
Further information about the HeartAI HIB interface service may be found with the following documentation:
Kiali UI application graph
The following image shows an example of the Kiali application graph web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod
namespace. The Kiali application graph interface provides:
- Observability of service mesh traffic.
- Indications of traffic status and volume.
- Service mesh components:
- The service mesh gateway
- The service mesh reverse-proxy
- PostgreSQL (
postgresql
) as a backing service. - Apache Kafka (
kafka-brokers
) as a backing service.
Kiali UI service graph
The following image shows an example of the Kiali service graph web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod
namespace. The Kiali service graph interface provides:
- Observability of service mesh traffic.
- Indications of traffic status and volume.
- Service mesh components:
- The service mesh gateway
- The service mesh reverse-proxy
- PostgreSQL (
postgresql
) as a backing service. - Apache Kafka (
kafka-brokers
) as a backing service.
Kiali UI application overview
The following image shows an example of the Kiali application overview web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod
namespace. The Kiali application overview interface provides:
- The workloads and services of the service mesh.
- An overview of the service mesh topology.
- An overview of the service mesh health.
Kiali UI application traffic
The following image shows an example of the Kiali application traffic web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod
namespace. The Kiali application traffic interface provides:
- An overview of inbound traffic, including:
- The current status of the service mesh.
- The source of inbound traffic.
- The inbound traffic protocol type.
- Metrics relating to traffic volume and status.
- An overview of outbound traffic, including:
- The current status of the service mesh.
- The destination of outbound traffic.
- The outbound traffic protocol type.
- Metrics relating to traffic volume and status.
Kiali UI application inbound metrics
The following image shows an example of the Kiali inbound metrics web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod
namespace. The Kiali inbound metrics interface provides:
- Visualisation of service mesh inbound metrics, including:
- Request volume.
- Request duration.
- Request throughput.
- Request size.
- Response throughput.
- Response size.
- TCP received.
- TCP size.
Kiali UI application outbound metrics
The following image shows an example of the Kiali outbound metrics web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod
namespace. The Kiali outbound metrics interface provides:
- Visualisation of service mesh outbound metrics, including:
- Request volume.
- Request duration.
- Request throughput.
- Request size.
- Response throughput.
- Response size.
- TCP received.
- TCP size.
Kiali UI application traces
The following image shows an example of the Kiali application traces web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod
namespace. The Kiali application traces interface provides:
- Visualisations for distributed tracing of the service mesh, including:
- Comparisons of service mesh trace response duration over time.
- Filtering by response status code where applicable.
Cloudflare implementation
For the components of HeartAI that are hosted on the public internet (this website and the Postman API reference), all traffic is proxied through Cloudflare as an edge network.
The following image shows an overview of analytics collected over a one-week period for connections that are proxied through Cloudflare: