Network architecture

Network architecture overview

HeartAI deploys to virtual networks with implementations that are provided by Microsoft Azure. Network instances typically deploy through a hub-spoke network topology with the hub instance implemented with Azure Virtual WAN (vWAN) with connectivity through virtual network peering between the virtual network and Azure vWAN hub instances. This allows HeartAI to deploy into existing organisational network topologies that implement an Azure vWAN hub with a well-defined connectivity mechanism. HeartAI network implementations are often private networks, and with the additional application of virtual network traffic routing this enforces network traffic to route through the corresponding organisational network topology to ensure that appropriate logging, monitoring, and access control is achieved.

HeartAI deploys supporting functionalities including Azure Private DNS to provide internal name resolution, Azure Network Watcher for logging and monitoring of network instances, and Azure Network security groups for network traffic control. HeartAI services do not expose network endpoints to the public internet - Service network resolution occurs internal to the HeartAI network or through private network extension.

HeartAI instances of Red Hat OpenShift provide native support for network management through first-class software-defined networking (SDN). This allows the OpenShift environment to operate as a virtual private cloud with OpenShift namespaces that are effectively separated from each other by tunnelled overlay network VXLAN. These implementations are natively provided by OpenShift integration with Open vSwitch. OpenShift allows location transparent name resolution to occur across a distributed cluster with name resolution provided with integrated CoreDNS instances. HeartAI typically deploys OpenShift through Microsoft Azure Red Hat OpenShift, such that OpenShift network instances are themselves virtualised as overlay networks on top of Azure virtual network instances.

HeartAI services are accessible through OpenShift service endpoint routes that are resolvable within the HeartAI domain and through the domains of extending networks. These endpoint routes provide network proxy to corresponding load-balanced reverse-proxies that are implemented with OpenShift services.

In addition to the SDN capabilities provided by OpenShift, HeartAI networking also implements OpenShift Service Mesh, providing advanced mechanisms for communication across services. The cloud-native service-mesh software Istio extends the OpenShift SDN with programmable and application-aware declarative network implementations. A core feature of Istio is the Envoy service proxy that is injectable as a sidecar into virtual IP hosts of the OpenShift SDN. Istio provides general approaches for network deployments, routing, traffic management, telemetry, and security. The management console Kiali provides capabilities for configuration, eventing, metrics, visualisation, and validation of network deployments that are implemented with Istio. Kiali allows for the display of service mesh structure by inferring traffic topology and health status. Kiali also provides native integration for the Grafana observability platform, and the Jaeger distributed tracing software.

Encryption over TLS occurs wherever possible for internal and external network communication. For example, service instances may communicate with each other via TCP over TLS, HTTPS, or WebSockets Secure. Similarly, backing services, such as persistent data stores, communicate with service instances through these mechanisms. Encryption also occurs for communication within OpenShift environments between the system control plane and compute nodes as well as between system components. The combined functionality of these approaches allows HeartAI network instances to be implemented as zero trust networks. The HeartAI OpenShift environment provides automated approaches for certificate issuance and management. This is primarily managed with the certificate management software cert-manager. cert-manager automates the issuance of certificates by utilising the ACME protocol, which also supports certificate rotation and revocation.

Domain name architecture

Further information about the HeartAI domain name architecture may be found with the following documentation:

System infrastructure

Further information about the HeartAI system infrastructure may be found with the following documentation:

Azure network architecture

Azure Virtual Network provides cloud-hosted networking infrastructure. HeartAI services do not expose network endpoints to the public internet - All network resolution occurs internal to the HeartAI network or through private network extension.

The following example shows a typical HeartAI Azure network implementation with two Azure Virtual Network instances. The first network is specific for a HeartAI production environment instance while the second network is specific for a HeartAI testing environment instance.

HeartAI Azure Virtual Network address spaces

Network name Description Network address range Hosting network
sah-heartai-vnet-prod-aue-001 HeartAI production environment 10.X.X.0/24 Microsoft Azure
sah-heartai-vnet-test-aue-001 HeartAI testing environment 10.X.X.0/24 Microsoft Azure

The following example shows the partitioning of a HeartAI production environment Azure Virtual Network instance to the following subnetworks:

HeartAI production environment subnetworks

Subnet name Description Subnet address Address range Available IPs Hosts
sah-heartai-snet-aroworker-prod-aue-001 Azure Red Hat Openshift worker nodes 10.X.X.0/25 10.X.X.0 - 10.X.X.127 10.X.X.1 - 10.X.X.126 126
sah-heartai-snet-aromaster-prod-aue-002 Azure Red Hat Openshift master nodes 10.X.X.128/27 10.X.X.128 - 10.X.X.159 10.X.X.129 - 10.X.X.158 30
Subnet range unassigned 10.X.X.160/27 10.X.X.160 - 10.X.X.191 10.X.X.161 - 10.X.X.190 30
sah-heartai-snet-paas-prod-aue-004 Azure PaaS endpoints 10.X.X.192/26 10.X.X.192 - 10.X.X.255 10.X.X.193 - 10.X.X.254 62

Terraform declaration

Terraform implementation

The HeartAI implementation of Microsoft Azure is managed with the Terraform declarative infrastructure-as-code software framework. Terraform allows for the declaration of system components using configuration files specified with the HashiCorp Configuration Language (HCL). Further information about the HeartAI implementation of Terraform may be found with the following documentation:

The following example shows Terraform declarations to specify an Azure Virtual Network. This implementation coordinates instances of the following Azure components:

Azure component Functionality for Azure Virtual Network
Private DNS Provides private name resolution for Azure Storage Account network endpoints within the corresponding Azure Private DNS zone
Virtual Network Configures and deploys an Azure Virtual Network
Network Watcher Integrates an instance of Azure Network Watcher to a corresponding Azure Virtual Network
resource "azurerm_private_dns_zone" "heartai-dns-private" {
  name = "sah.heartai.net"
  resource_group_name = azurerm_resource_group.rg.name

  tags = {
    "Application Name" = var.heartai-environment.application-name
    "Application Owner" = var.heartai-environment.application-owner
    "Environment" = var.heartai-environment.environment
    "Division Department" = var.heartai-environment.division-department
    "Cost Centre" = var.heartai-environment.cost-centre
  }
}

resource "azurerm_virtual_network" "vnet" {
  name = "sah-heartai-vnet-prod-aue-001"
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  address_space = [
    var.vnet-prod-aue-001-address-space.vnet]

  tags = {
    "Application Name" = var.heartai-environment.application-name
    "Application Owner" = var.heartai-environment.application-owner
    "Environment" = var.heartai-environment.environment
    "Division Department" = var.heartai-environment.division-department
    "Cost Centre" = var.heartai-environment.cost-centre
  }
}

resource "azurerm_subnet" "snet_aroworker" {
  name = "sah-heartai-snet-aroworker-prod-aue-001"
  resource_group_name = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes = [
    var.vnet-prod-aue-001-address-space.snet1]
  service_endpoints = [
    "Microsoft.ContainerRegistry"]
  enforce_private_link_endpoint_network_policies = true
}

resource "azurerm_subnet" "snet_aromaster" {
  name = "sah-heartai-snet-aromaster-prod-aue-002"
  resource_group_name = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes = [
    var.vnet-prod-aue-001-address-space.snet2]
  service_endpoints = [
    "Microsoft.ContainerRegistry"]
  enforce_private_link_service_network_policies = true
}

resource "azurerm_subnet" "snet_vpn_gateway" {
  name = "GatewaySubnet"
  resource_group_name = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes = [
    var.vnet-prod-aue-001-address-space.snet3]
  enforce_private_link_endpoint_network_policies = true
}

resource "azurerm_subnet" "snet_paas" {
  name = "sah-heartai-snet-paas-prod-aue-004"
  resource_group_name = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes = [
    var.vnet-prod-aue-001-address-space.snet4]
  enforce_private_link_endpoint_network_policies = true
}

resource "azurerm_network_watcher" "network_watcher" {
  name = "NetworkWatcher_australiaeast"
  location = "australiaeast"
  resource_group_name = "NetworkWatcherRG"

  tags = {
    "Application Name" = var.heartai-environment.application-name
    "Application Owner" = var.heartai-environment.application-owner
    "Environment" = var.heartai-environment.environment
    "Division Department" = var.heartai-environment.division-department
    "Cost Centre" = var.heartai-environment.cost-centre
  }
}

Azure deployment overview

The following figure shows a structural overview of Microsoft Azure cloud resources within a HeartAI production environment instance. This figures represents:

  • A corresponding Azure vWAN hub, including:
    • An Azure ExpressRoute as an example of external network connectivity.
    • An Azure Virtual WAN instance.
    • Network peering between a HeartAI Azure Virtual Network instance and a corresponding Azure vWAN hub.
  • A HeartAI Azure Virtual Network instance, with the following contained resources:
    • Azure Red Hat OpenShift Master nodes.
    • Azure Red Hat OpenShift Worker nodes.
    • Azure private endpoints, with internal and private network connectivity to Azure cloud services.
  • Azure cloud services, including:

heartai-azure-network-architecture.svg

Approaches for access to organisational networks

HeartAI end-users, developers, and administrators may access on-premises network instances and Azure tenancies in coordination with a hosting organisation. Access typically occurs through the three following approaches:

Azure network access approach Description Network access type
Organisation foreign network point-to-site VPN Point-to-site network access from an end-users gateway to the foreign network VPN gateway of a hosting organisation. On-premises point-to-site VPN
Azure Virtual WAN point-to-site VPN Point-to-site network access from an end-users gateway to an Azure VPN Gateway within an Azure Virtual WAN instance of a hosting organisation. Cloud point-to-site VPN
Azure Virtual Network point-to-site VPN Point-to-site network access from an end-users gateway to an Azure VPN Gateway within an Azure Virtual Network instance of a hosting organisation. Cloud point-to-site VPN

Example: Organisation foreign network point-to-site VPN

The following figure shows an example organisation foreign network point-to-site VPN, representing access from an end-user device to the foreign network gateway of a hosting organisation:

heartai-system-network-architecture-point-to-site-network-access-organisation-foreign-network-vpn-gateway-deployment.svg

Example: Azure Virtual WAN point-to-site VPN

The following figure shows an example Azure Virtual WAN point-to-site VPN, representing access from an end-user device to the Azure VPN Gateway within an Azure Virtual WAN instance of a hosting organisation:

heartai-system-network-architecture-point-to-site-network-access-azure-vwan-vpn-gateway-deployment.svg

Example: Azure Virtual Network point-to-site VPN

The following figure shows an example Azure Virtual Network point-to-site VPN, representing access from an end-user device to the Azure VPN Gateway within an Azure Virtual Network instance of a hosting organisation:

heartai-system-network-architecture-point-to-site-network-access-azure-vnet-vpn-gateway-deployment.svg

Organisational DNS zone architecture

For deployments within hosting organisational networks, HeartAI typically requests to manage a private DNZ zone. In the most direct DNS zone architecture deployment, the following DNZ zones are defined:

DNZ zone Description Managing organisation Corresponding network
[hostorg].[tld] An organisational DNS zone. Hosting organisation An organisational network.
[hostorg].heartai.net An instance of Azure Private DNS zone. HeartAI A private Azure Virtual Network as a private extension of the organisation network.
Domain name architecture

Further information about HeartAI domain name architecture may be found with the following:

The following example shows a typical DNS zone architecture within a hosting organisational network structure. The corresponding organisation and HeartAI DNS zones are represented with the network constructs between these environments.

heartai-system-network-architecture-organisation-dns-zone-architecture.svg

HeartAI private DNS orchestration within a deployed environment is also important for the automation of cryptographic certificates.

Within HeartAI OpenShift environment provides automated approaches for certificate issuance and management. This is primarily coordinated with the certificate management software cert-manager. cert-manager automates the issuance of certificates by utilising the ACME protocol, which also supports certificate rotation and revocation. The corresponding certificate authority is Let’s Encrypt. The preferred challenge mechanism to prove HeartAI domain ownership is with the DNS01 challenge. cert-manager performs this challenge by communicating with the HeartAI global domain name servers that are implemented with Cloudflare DNS. This allows HeartAI domain hostnames to issue trustable certificates.

Network security

Further information about HeartAI network security may be found with the following:

OpenShift network architecture

HeartAI instances of Red Hat OpenShift provide native support for network management through first-class software-defined networking (SDN). This allows the OpenShift environment to operate as a virtual private cloud with OpenShift namespaces that are effectively separated from each other by tunnelled overlay network VXLAN. These implementations are natively provided by OpenShift integration with Open vSwitch. OpenShift allows location transparent name resolution to occur across a distributed cluster with name resolution provided with integrated CoreDNS instances. HeartAI typically deploys OpenShift through Microsoft Azure Red Hat OpenShift, such that OpenShift network instances are themselves virtualised as overlay networks on top of Azure virtual network instances.

OpenShift service endpoint architecture

HeartAI services are accessible through OpenShift service endpoint routes that are resolvable within the HeartAI domain and through the domains of extending networks. These endpoint routes provide network proxy to corresponding load-balanced reverse-proxies that are implemented with OpenShift services.

heartai-service-endpoint-network-architecture.svg

OpenShift client endpoint architecture

System end-users may also access the system through clients that act on behalf of the end-user. Network access to these clients is similar to the direct services access approaches described above. System protected resources are accessible to these clients provided that the end-user has delegated the corresponding authority to the client for these purposes.

heartai-client-endpoint-network-architecture.svg

Identity and access management

Please refer to the documentation section Identity and access management for an overview of these approaches.

OpenShift proxied client endpoint architecture

In addition to client network resolution, clients may be resolved through sidecar network reverse-proxy. The primary implementation with this approach is through OAuth 2 Proxy. This approach is particularly useful for legacy clients that have a limited ability to independently perform the OAuth 2.0 Authorization Code Grant.

heartai-proxied-client-endpoint-network-architecture.svg

Identity and access management

Please refer to the documentation section Identity and access management for an overview of these approaches.

OpenShift Service Mesh implementation

In addition to the SDN capabilities provided by OpenShift, HeartAI networking also implements OpenShift Service Mesh, providing advanced mechanisms for communication across services. The cloud-native service-mesh software Istio extends the OpenShift SDN with programmable and application-aware declarative network implementations. A core feature of Istio is the Envoy service proxy that is injectable as a sidecar into virtual IP hosts of the OpenShift SDN. Istio provides general approaches for network deployments, routing, traffic management, telemetry, and security. The management console Kiali provides capabilities for configuration, eventing, metrics, visualisation, and validation of network deployments that are implemented with Istio. Kiali allows for the display of service mesh structure by inferring traffic topology and health status. Kiali also provides native integration for the Grafana observability platform, and the Jaeger distributed tracing software.

OpenShift Service Mesh composes the following technologies:

OpenShift Service Mesh technology Description Reference
Istio Modern service mesh that is natively implemented with Kubernetes networking capabilities. Supports advanced network routing and monitoring through the use of injectable sidecar proxies. https://istio.io/
Kiali Management console for observability and management of Istio implementations. https://kiali.io/
Jaeger End-to-end distributed tracing. First-class integration with Istio and Kiali. https://www.jaegertracing.io/

Istio implementation

The cloud-native service-mesh software Istio extends the OpenShift SDN with programmable and application-aware declarative network implementations. A core feature of Istio is the Envoy service proxy that is injectable as a sidecar into virtual IP hosts of the OpenShift SDN. Istio provides general approaches for network deployments, routing, traffic management, telemetry, and security.

Istio implementation

Further information about the HeartAI Istio implementation may be found with the following documentation:

Example implementation components

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: heartai-hello-world-prod-gw
  namespace: heartai-hello-world-prod
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      tls:
        httpsRedirect: true
      hosts:
        - hello.prod.apps.aro.sah.heartai.net
    - port:
        number: 443
        name: https
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: heartai-hello-world-prod-gw-cert
      hosts:
        - hello.prod.apps.aro.sah.heartai.net

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: heartai-hello-world-vs
  namespace: heartai-hello-world-prod
spec:
  hosts:
    - hello.prod.apps.aro.sah.heartai.net
  gateways:
    - heartai-hello-world-prod-gw
  http:
    - match:
        - uri:
            prefix: "/"
      route:
        - destination:
            host: heartai-hello-world

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: heartai-hello-world-nw-rule
  namespace: heartai-hello-world-prod
spec:
  host: heartai-hello-world
  subsets:
    - name: stable
      labels:
        version: v0.31.106

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: postgresql
  namespace: heartai-hello-world-prod
spec:
  hosts:
    - sah-heartai-psql-prod-aue-001.postgres.database.azure.com
  ports:
    - number: 5432
      name: postgresql
      protocol: tcp
  resolution: DNS
  location: MESH_EXTERNAL

Kiali implementation

Kiali provides capabilities for configuration, eventing, metrics, visualisation, and validation of Istio service mesh software. Kiali allows for the display of service mesh structure by inferring traffic topology and health status, and supports reporting and visualisation with:

  • Graphs displaying service mesh topology and real-time traffic flow.
  • Service mesh status and health updates.
  • An overview of service mesh components, including associated workloads and services.
  • Reports and status checks for inbound and outbound traffic.
  • Metrics and visualisation for inbound and outbound traffic.
  • Tooling for distributed service mesh tracing.

HeartAI instances of Red Hat OpenShift implementation are extendable with OpenShift Service Mesh, integrating Istio, Kiali, and the Jaeger distributed tracing software.

Kiali implementation

Further information about the HeartAI Kiali implementation may be found with the following documentation:

The following image shows an example of the Kiali management console with an implementation of Istio for the heartai-hello-world-dev namespace. The components of this namespace relate to the HeartAI HelloWorldService instance and show the service gateway and reverse-proxy, and the backing services PostgreSQL (postgresql) and Apache Kafka (kafka-brokers):

heartai-istio-kiali.png

The following images are from the HeartAI interface with the SA Health Health Information Broker, a major provider of HL7 formatted health data within SA Health:

Services: HIB interface

Further information about the HeartAI HIB interface service may be found with the following documentation:

Kiali UI application graph

The following image shows an example of the Kiali application graph web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali application graph interface provides:

  • Observability of service mesh traffic.
  • Indications of traffic status and volume.
  • Service mesh components:
    • The service mesh gateway
    • The service mesh reverse-proxy
    • PostgreSQL (postgresql) as a backing service.
    • Apache Kafka (kafka-brokers) as a backing service.

heartai-istio-kiali-app-graph-hib-interface-prod.png

Kiali UI service graph

The following image shows an example of the Kiali service graph web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali service graph interface provides:

  • Observability of service mesh traffic.
  • Indications of traffic status and volume.
  • Service mesh components:
    • The service mesh gateway
    • The service mesh reverse-proxy
    • PostgreSQL (postgresql) as a backing service.
    • Apache Kafka (kafka-brokers) as a backing service.

heartai-istio-kiali-service-graph-hib-interface-prod.png

Kiali UI application overview

The following image shows an example of the Kiali application overview web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali application overview interface provides:

  • The workloads and services of the service mesh.
  • An overview of the service mesh topology.
  • An overview of the service mesh health.

heartai-istio-kiali-application-overview-hib-interface-prod.png

Kiali UI application traffic

The following image shows an example of the Kiali application traffic web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali application traffic interface provides:

  • An overview of inbound traffic, including:
    • The current status of the service mesh.
    • The source of inbound traffic.
    • The inbound traffic protocol type.
    • Metrics relating to traffic volume and status.
  • An overview of outbound traffic, including:
    • The current status of the service mesh.
    • The destination of outbound traffic.
    • The outbound traffic protocol type.
    • Metrics relating to traffic volume and status.

heartai-istio-kiali-application-traffic-hib-interface-prod.png

Kiali UI application inbound metrics

The following image shows an example of the Kiali inbound metrics web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali inbound metrics interface provides:

  • Visualisation of service mesh inbound metrics, including:
    • Request volume.
    • Request duration.
    • Request throughput.
    • Request size.
    • Response throughput.
    • Response size.
    • TCP received.
    • TCP size.

heartai-istio-kiali-application-inbound-metrics-hib-interface-prod.png

Kiali UI application outbound metrics

The following image shows an example of the Kiali outbound metrics web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali outbound metrics interface provides:

  • Visualisation of service mesh outbound metrics, including:
  • Request volume.
  • Request duration.
  • Request throughput.
  • Request size.
  • Response throughput.
  • Response size.
  • TCP received.
  • TCP size.

heartai-istio-kiali-application-outbound-metrics-hib-interface-prod.png

Kiali UI application traces

The following image shows an example of the Kiali application traces web interface corresponding to an implementation of Istio for the heartai-hib-interface-prod namespace. The Kiali application traces interface provides:

  • Visualisations for distributed tracing of the service mesh, including:
    • Comparisons of service mesh trace response duration over time.
    • Filtering by response status code where applicable.

heartai-istio-kiali-application-tracing-hib-interface-prod.png

Cloudflare implementation

For the components of HeartAI that are hosted on the public internet (this website and the Postman API reference), all traffic is proxied through Cloudflare as an edge network.

The following image shows an overview of analytics collected over a one-week period for connections that are proxied through Cloudflare:

heartai-cloudflare-all-pages-analytics.png