Network architecture

Network architecture overview

HeartAI deploys to virtual networks with implementations that are provided by Microsoft Azure. Network instances typically deploy through a hub-spoke network topology with the hub instance implemented with Azure Virtual WAN (vWAN) with connectivity through virtual network peering between the virtual network and Azure vWAN hub instances. This allows HeartAI to deploy into existing organisational network topologies that implement an Azure vWAN hub with a well-defined connectivity mechanism. HeartAI network implementations are often private networks, and with the additional application of virtual network traffic routing this enforces network traffic to route through the corresponding organisational network topology to ensure that appropriate logging, monitoring, and access control is achieved.

HeartAI deploys supporting functionalities including Azure Private DNS to provide internal name resolution, Azure Network Watcher for logging and monitoring of network instances, and Azure Network security groups for network traffic control. HeartAI services do not expose network endpoints to the public internet - Service network resolution occurs internal to the HeartAI network or through private network extension.

HeartAI instances of Red Hat OpenShift provide native support for network management through first-class software-defined networking (SDN). This allows the OpenShift environment to operate as a virtual private cloud with OpenShift namespaces that are effectively separated from each other by tunnelled overlay network VXLAN. These implementations are natively provided by OpenShift integration with Open vSwitch. OpenShift allows location transparent name resolution to occur across a distributed cluster with name resolution provided with integrated CoreDNS instances. HeartAI typically deploys OpenShift through Microsoft Azure Red Hat OpenShift, such that OpenShift network instances are themselves virtualised as overlay networks on top of Azure virtual network instances.

HeartAI services are accessible through OpenShift service endpoint routes that are resolvable within the HeartAI domain and through the domains of extending networks. These endpoint routes provide network proxy to corresponding load-balanced reverse-proxies that are implemented with OpenShift services.

In addition to the SDN capabilities provided by OpenShift, HeartAI networking also implements OpenShift Service Mesh, providing advanced mechanisms for communication across services. The cloud-native service-mesh software Istio extends the OpenShift SDN with programmable and application-aware declarative network implementations. A core feature of Istio is the Envoy service proxy that is injectable as a sidecar into virtual IP hosts of the OpenShift SDN. Istio provides general approaches for network deployments, routing, traffic management, telemetry, and security. The management console Kiali provides capabilities for configuration, eventing, metrics, visualisation, and validation of network deployments that are implemented with Istio. Kiali allows for the display of service mesh structure by inferring traffic topology and health status. Kiali also provides native integration for the Grafana observability platform, and the Jaeger distributed tracing software.

Encryption over TLS occurs wherever possible for internal and external network communication. For example, service instances may communicate with each other via TCP over TLS, HTTPS, or WebSockets Secure. Similarly, backing services, such as persistent data stores, communicate with service instances through these mechanisms. Encryption also occurs for communication within OpenShift environments between the system control plane and compute nodes as well as between system components. The combined functionality of these approaches allows HeartAI network instances to be implemented as zero trust networks. The HeartAI OpenShift environment provides automated approaches for certificate issuance and management. This is primarily managed with the certificate management software cert-manager. cert-manager automates the issuance of certificates by utilising the ACME protocol, which also supports certificate rotation and revocation.

Azure network architecture

Azure Virtual Network provides cloud-hosted networking infrastructure. HeartAI services do not expose network endpoints to the public internet - All network resolution occurs internal to the HeartAI network or through private network extension.

The following example shows a typical HeartAI Azure network implementation with two Azure Virtual Network instances. The first network is specific for a HeartAI production environment instance while the second network is specific for a HeartAI testing environment instance.

HeartAI Azure Virtual Network address spaces

Network name	Description	Network address range	Hosting network
sah-heartai-vnet-prod-aue-001	HeartAI production environment	10.X.X.0/24	Microsoft Azure
sah-heartai-vnet-test-aue-001	HeartAI testing environment	10.X.X.0/24	Microsoft Azure

The following example shows the partitioning of a HeartAI production environment Azure Virtual Network instance to the following subnetworks:

HeartAI production environment subnetworks

Subnet name	Description	Subnet address	Address range	Available IPs	Hosts
sah-heartai-snet-aroworker-prod-aue-001	Azure Red Hat Openshift worker nodes	10.X.X.0/25	10.X.X.0 - 10.X.X.127	10.X.X.1 - 10.X.X.126	126
sah-heartai-snet-aromaster-prod-aue-002	Azure Red Hat Openshift master nodes	10.X.X.128/27	10.X.X.128 - 10.X.X.159	10.X.X.129 - 10.X.X.158	30
	Subnet range unassigned	10.X.X.160/27	10.X.X.160 - 10.X.X.191	10.X.X.161 - 10.X.X.190	30
sah-heartai-snet-paas-prod-aue-004	Azure PaaS endpoints	10.X.X.192/26	10.X.X.192 - 10.X.X.255	10.X.X.193 - 10.X.X.254	62

Terraform declaration

The following example shows Terraform declarations to specify an Azure Virtual Network. This implementation coordinates instances of the following Azure components:

Azure component	Functionality for Azure Virtual Network
Private DNS	Provides private name resolution for Azure Storage Account network endpoints within the corresponding Azure Private DNS zone
Virtual Network	Configures and deploys an Azure Virtual Network
Network Watcher	Integrates an instance of Azure Network Watcher to a corresponding Azure Virtual Network

copyresource "azurerm_private_dns_zone" "heartai-dns-private" {
  name = "sah.heartai.net"
  resource_group_name = azurerm_resource_group.rg.name

  tags = {
    "Application Name" = var.heartai-environment.application-name
    "Application Owner" = var.heartai-environment.application-owner
    "Environment" = var.heartai-environment.environment
    "Division Department" = var.heartai-environment.division-department
    "Cost Centre" = var.heartai-environment.cost-centre
  }
}

resource "azurerm_virtual_network" "vnet" {
  name = "sah-heartai-vnet-prod-aue-001"
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  address_space = [
    var.vnet-prod-aue-001-address-space.vnet]

  tags = {
    "Application Name" = var.heartai-environment.application-name
    "Application Owner" = var.heartai-environment.application-owner
    "Environment" = var.heartai-environment.environment
    "Division Department" = var.heartai-environment.division-department
    "Cost Centre" = var.heartai-environment.cost-centre
  }
}

resource "azurerm_subnet" "snet_aroworker" {
  name = "sah-heartai-snet-aroworker-prod-aue-001"
  resource_group_name = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes = [
    var.vnet-prod-aue-001-address-space.snet1]
  service_endpoints = [
    "Microsoft.ContainerRegistry"]
  enforce_private_link_endpoint_network_policies = true
}

resource "azurerm_subnet" "snet_aromaster" {
  name = "sah-heartai-snet-aromaster-prod-aue-002"
  resource_group_name = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes = [
    var.vnet-prod-aue-001-address-space.snet2]
  service_endpoints = [
    "Microsoft.ContainerRegistry"]
  enforce_private_link_service_network_policies = true
}

resource "azurerm_subnet" "snet_vpn_gateway" {
  name = "GatewaySubnet"
  resource_group_name = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes = [
    var.vnet-prod-aue-001-address-space.snet3]
  enforce_private_link_endpoint_network_policies = true
}

resource "azurerm_subnet" "snet_paas" {
  name = "sah-heartai-snet-paas-prod-aue-004"
  resource_group_name = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes = [
    var.vnet-prod-aue-001-address-space.snet4]
  enforce_private_link_endpoint_network_policies = true
}

resource "azurerm_network_watcher" "network_watcher" {
  name = "NetworkWatcher_australiaeast"
  location = "australiaeast"
  resource_group_name = "NetworkWatcherRG"

  tags = {
    "Application Name" = var.heartai-environment.application-name
    "Application Owner" = var.heartai-environment.application-owner
    "Environment" = var.heartai-environment.environment
    "Division Department" = var.heartai-environment.division-department
    "Cost Centre" = var.heartai-environment.cost-centre
  }
}

Azure deployment overview

The following figure shows a structural overview of Microsoft Azure cloud resources within a HeartAI production environment instance. This figures represents:

• A corresponding Azure vWAN hub, including:
  • An Azure ExpressRoute as an example of external network connectivity.
  • An Azure Virtual WAN instance.
  • Network peering between a HeartAI Azure Virtual Network instance and a corresponding Azure vWAN hub.
• A HeartAI Azure Virtual Network instance, with the following contained resources:
  • Azure Red Hat OpenShift Master nodes.
  • Azure Red Hat OpenShift Worker nodes.
  • Azure private endpoints, with internal and private network connectivity to Azure cloud services.
• Azure cloud services, including:
  • Azure Key Vault.
  • Azure Sentinel.
  • Azure Monitor.
  • Azure Storage.
  • Azure Database for PostgreSQL.

Approaches for access to organisational networks

HeartAI end-users, developers, and administrators may access on-premises network instances and Azure tenancies in coordination with a hosting organisation. Access typically occurs through the three following approaches:

Azure network access approach	Description	Network access type
Organisation foreign network point-to-site VPN	Point-to-site network access from an end-users gateway to the foreign network VPN gateway of a hosting organisation.	On-premises point-to-site VPN
Azure Virtual WAN point-to-site VPN	Point-to-site network access from an end-users gateway to an Azure VPN Gateway within an Azure Virtual WAN instance of a hosting organisation.	Cloud point-to-site VPN
Azure Virtual Network point-to-site VPN	Point-to-site network access from an end-users gateway to an Azure VPN Gateway within an Azure Virtual Network instance of a hosting organisation.	Cloud point-to-site VPN

Example: Organisation foreign network point-to-site VPN

The following figure shows an example organisation foreign network point-to-site VPN, representing access from an end-user device to the foreign network gateway of a hosting organisation:

Example: Azure Virtual WAN point-to-site VPN

The following figure shows an example Azure Virtual WAN point-to-site VPN, representing access from an end-user device to the Azure VPN Gateway within an Azure Virtual WAN instance of a hosting organisation:

Example: Azure Virtual Network point-to-site VPN

The following figure shows an example Azure Virtual Network point-to-site VPN, representing access from an end-user device to the Azure VPN Gateway within an Azure Virtual Network instance of a hosting organisation:

Organisational DNS zone architecture

For deployments within hosting organisational networks, HeartAI typically requests to manage a private DNZ zone. In the most direct DNS zone architecture deployment, the following DNZ zones are defined:

DNZ zone	Description	Managing organisation	Corresponding network
[hostorg].[tld]	An organisational DNS zone.	Hosting organisation	An organisational network.
[hostorg].heartai.net	An instance of Azure Private DNS zone.	HeartAI	A private Azure Virtual Network as a private extension of the organisation network.

The following example shows a typical DNS zone architecture within a hosting organisational network structure. The corresponding organisation and HeartAI DNS zones are represented with the network constructs between these environments.

HeartAI private DNS orchestration within a deployed environment is also important for the automation of cryptographic certificates.

Within HeartAI OpenShift environment provides automated approaches for certificate issuance and management. This is primarily coordinated with the certificate management software cert-manager. cert-manager automates the issuance of certificates by utilising the ACME protocol, which also supports certificate rotation and revocation. The corresponding certificate authority is Let’s Encrypt. The preferred challenge mechanism to prove HeartAI domain ownership is with the DNS01 challenge. cert-manager performs this challenge by communicating with the HeartAI global domain name servers that are implemented with Cloudflare DNS. This allows HeartAI domain hostnames to issue trustable certificates.

OpenShift network architecture

HeartAI instances of Red Hat OpenShift provide native support for network management through first-class software-defined networking (SDN). This allows the OpenShift environment to operate as a virtual private cloud with OpenShift namespaces that are effectively separated from each other by tunnelled overlay network VXLAN. These implementations are natively provided by OpenShift integration with Open vSwitch. OpenShift allows location transparent name resolution to occur across a distributed cluster with name resolution provided with integrated CoreDNS instances. HeartAI typically deploys OpenShift through Microsoft Azure Red Hat OpenShift, such that OpenShift network instances are themselves virtualised as overlay networks on top of Azure virtual network instances.

OpenShift service endpoint architecture

HeartAI services are accessible through OpenShift service endpoint routes that are resolvable within the HeartAI domain and through the domains of extending networks. These endpoint routes provide network proxy to corresponding load-balanced reverse-proxies that are implemented with OpenShift services.

OpenShift client endpoint architecture

System end-users may also access the system through clients that act on behalf of the end-user. Network access to these clients is similar to the direct services access approaches described above. System protected resources are accessible to these clients provided that the end-user has delegated the corresponding authority to the client for these purposes.

OpenShift proxied client endpoint architecture

In addition to client network resolution, clients may be resolved through sidecar network reverse-proxy. The primary implementation with this approach is through OAuth 2 Proxy. This approach is particularly useful for legacy clients that have a limited ability to independently perform the OAuth 2.0 Authorization Code Grant.

OpenShift Service Mesh implementation

In addition to the SDN capabilities provided by OpenShift, HeartAI networking also implements OpenShift Service Mesh, providing advanced mechanisms for communication across services. The cloud-native service-mesh software Istio extends the OpenShift SDN with programmable and application-aware declarative network implementations. A core feature of Istio is the Envoy service proxy that is injectable as a sidecar into virtual IP hosts of the OpenShift SDN. Istio provides general approaches for network deployments, routing, traffic management, telemetry, and security. The management console Kiali provides capabilities for configuration, eventing, metrics, visualisation, and validation of network deployments that are implemented with Istio. Kiali allows for the display of service mesh structure by inferring traffic topology and health status. Kiali also provides native integration for the Grafana observability platform, and the Jaeger distributed tracing software.

OpenShift Service Mesh composes the following technologies:

OpenShift Service Mesh technology	Description	Reference
Istio	Modern service mesh that is natively implemented with Kubernetes networking capabilities. Supports advanced network routing and monitoring through the use of injectable sidecar proxies.	https://istio.io/
Kiali	Management console for observability and management of Istio implementations.	https://kiali.io/
Jaeger	End-to-end distributed tracing. First-class integration with Istio and Kiali.	https://www.jaegertracing.io/

Istio implementation

The cloud-native service-mesh software Istio extends the OpenShift SDN with programmable and application-aware declarative network implementations. A core feature of Istio is the Envoy service proxy that is injectable as a sidecar into virtual IP hosts of the OpenShift SDN. Istio provides general approaches for network deployments, routing, traffic management, telemetry, and security.

Example implementation components

• Istio Gateway

copyapiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: heartai-hello-gw
  namespace: heartai-hello-world-prod
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      tls:
        httpsRedirect: true
      hosts:
        - hello.prod.apps.aro.sah.heartai.net
    - port:
        number: 443
        name: https
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: heartai-hello-world-prod-gw-cert
      hosts:
        - hello.prod.apps.aro.sah.heartai.net

• Istio Virtual Service

copyapiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: heartai-service-hello-world-vs-prod
  namespace: heartai-hello-world-prod
spec:
  hosts:
    - heartai-service-hello-world-prod.heartai-hello-world-prod.svc.cluster.local
  gateways:
    - mesh
  #      - heartai-service-hello-world-gw
  http:
    - match:
        - uri:
            prefix: "/"
      route:
        - destination:
            host: heartai-service-hello-world-prod

• Istio Destination Rule

copyapiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: heartai-service-hello-world-rule-prod
  namespace: heartai-hello-world-prod
spec:
  host: heartai-service-hello-world-prod
  subsets:
    - name: stable
      labels:
        version: v0.33.44

• Istio Service Entry

copyapiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: postgresql
  namespace: heartai-hello-world-prod
spec:
  hosts:
    - sah-heartai-psql-prod-aue-001.postgres.database.azure.com
  ports:
    - number: 5432
      name: postgresql
      protocol: tcp
  resolution: DNS
  location: MESH_EXTERNAL

Kiali implementation

Kiali provides capabilities for configuration, eventing, metrics, visualisation, and validation of Istio service mesh software. Kiali allows for the display of service mesh structure by inferring traffic topology and health status, and supports reporting and visualisation with:

• Graphs displaying service mesh topology and real-time traffic flow.
• Service mesh status and health updates.
• An overview of service mesh components, including associated workloads and services.
• Reports and status checks for inbound and outbound traffic.
• Metrics and visualisation for inbound and outbound traffic.
• Tooling for distributed service mesh tracing.

HeartAI instances of Red Hat OpenShift implementation are extendable with OpenShift Service Mesh, integrating Istio, Kiali, and the Jaeger distributed tracing software.

The following image shows an example of the Kiali management console with an implementation of Istio for the heartai-hello-world-dev namespace. The components of this namespace relate to the HeartAI HelloWorldService instance and show the service gateway and reverse-proxy, and the backing services PostgreSQL (postgresql) and Apache Kafka (kafka-brokers):

The following images are from the HeartAI interface with the SA Health Health Information Broker, a major provider of HL7 formatted health data within SA Health:

Kiali UI application graph

The following image shows an example of the Kiali application graph web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali application graph interface provides:

• Observability of service mesh traffic.
• Indications of traffic status and volume.
• Service mesh components:
  • The service mesh gateway
  • The service mesh reverse-proxy
  • PostgreSQL (postgresql) as a backing service.
  • Apache Kafka (kafka-brokers) as a backing service.

Kiali UI service graph

The following image shows an example of the Kiali service graph web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali service graph interface provides:

• Observability of service mesh traffic.
• Indications of traffic status and volume.
• Service mesh components:
  • The service mesh gateway
  • The service mesh reverse-proxy
  • PostgreSQL (postgresql) as a backing service.
  • Apache Kafka (kafka-brokers) as a backing service.

Kiali UI application overview

The following image shows an example of the Kiali application overview web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali application overview interface provides:

• The workloads and services of the service mesh.
• An overview of the service mesh topology.
• An overview of the service mesh health.

Kiali UI application traffic

The following image shows an example of the Kiali application traffic web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali application traffic interface provides:

• An overview of inbound traffic, including:
  • The current status of the service mesh.
  • The source of inbound traffic.
  • The inbound traffic protocol type.
  • Metrics relating to traffic volume and status.
• An overview of outbound traffic, including:
  • The current status of the service mesh.
  • The destination of outbound traffic.
  • The outbound traffic protocol type.
  • Metrics relating to traffic volume and status.

Kiali UI application inbound metrics

The following image shows an example of the Kiali inbound metrics web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali inbound metrics interface provides:

• Visualisation of service mesh inbound metrics, including:
  • Request volume.
  • Request duration.
  • Request throughput.
  • Request size.
  • Response throughput.
  • Response size.
  • TCP received.
  • TCP size.

Kiali UI application outbound metrics

The following image shows an example of the Kiali outbound metrics web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali outbound metrics interface provides:

• Visualisation of service mesh outbound metrics, including:
• Request volume.
• Request duration.
• Request throughput.
• Request size.
• Response throughput.
• Response size.
• TCP received.
• TCP size.

Kiali UI application traces

The following image shows an example of the Kiali application traces web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali application traces interface provides:

• Visualisations for distributed tracing of the service mesh, including:
  • Comparisons of service mesh trace response duration over time.
  • Filtering by response status code where applicable.

Cloudflare implementation

For the components of HeartAI that are hosted on the public internet (this website and the Postman API reference), all traffic is proxied through Cloudflare as an edge network.

The following image shows an overview of analytics collected over a one-week period for connections that are proxied through Cloudflare: