Amazon Managed Grafana and Amazon Managed Service for Prometheus

As organizations embrace agile approaches to development, adopting DevOps, CI/CD, and cloud-native technologies, deployment frequency increases. In fact, the DevOps Research and Assessment (DORA) group measures deployment frequency as a key maturity metric in its annual research. According to the group, elite performers deploy on-demand, often deploying multiple times per day. This volume of feature and service deployment requires observability to ensure ongoing application health. 

Observability uses system outputs to determine its health; the more outputs you can monitor and measure, the more accurate your understanding of system health. And, as the number of outputs grows, so does your ability (and confidence!) to quickly troubleshoot performance issues. 

Grafana and Prometheus observability

What started as an open-source project in 2014, Grafana has grown to become one of the most popular open-source projects on GitHub. It gained quick popularity as it helps users visualize complex data and metrics through easily shared dashboards. 

Also an open-source project, Prometheus was designed to monitor complex container environments, providing rich time series based metrics. Grafana built support for Prometheus metrics into its solution so that users could create visual dashboards for Prometheus metrics. Together, the solutions have become industry favorite open-source observability tools. 

While the combined solutions help grow observability, shrink troubleshooting and ensure system confidence, as with anything else, they require management and maintenance like scaling capacity, upgrades, back-ups and security – all of which add operational overhead. 

Amazon Managed Grafana and Amazon Managed Service for Prometheus

Enter the new Amazon Managed Grafana and Amazon Managed Service for Prometheus. As managed services, Amazon Web Services (AWS) removes operational overhead and streamlines management processes for users allowing them to focus instead on activities that add strategic value. Specifically, according to AWS, the two solutions: 

• Amazon Managed Grafana allows users to easily create, explore, and share observability dashboards, connecting to multiple data sources in the observability stack. AWS has connected Amazon Managed Grafana with its data sources like Amazon CloudWatch and Amazon Elasticsearch Service – and Amazon Managed Service for Prometheus, of course. It can also connect to solutions like Datadog, Splunk and InfluxDB. AWS also integrated Amazon Managed Grafana with AWS security services for secure data collection and analysis. To help you get started quickly, AWS provides pre-built dashboards.
  
  
• Users of Amazon Managed Service for Prometheus can continue to use Prometheus’s query language (PromQL) to monitor their container workload health but can now do so in the AWS environment where it will automatically scale in sync with workload needs. In addition, AWS integrates the solution with its security services for secure data access. It also integrates to AWS container services like Amazon Elastic Kubernetes Service (Amazon EKS) and Amazon Elastic Container Service (Amazon ECS), AMP allows you to easily collect Prometheus metrics from these environments.
  
  Both solutions aim to reduce operational costs by removing the resources required to run and manage Grafana and Prometheus at scale. 
  
  
  

AWS provides unified observability

AWS delivers on this promise, making it easier for users to monitor container metrics and observe AWS ecosystem health with Prometheus and Grafana. This will allow users to focus their resources on troubleshooting operational issues and improving application health, not management. And with connectivity to AWS container services and AWS monitoring services, users will be able to effortlessly monitor container health from a single dashboard.

Grafana and Prometheus in action

We recently helped a manufacturing client achieve multi-cluster monitoring in Amazon EKS with Prometheus and Grafana. The client had multiple production environments that were disconnected for legal compliance reasons. While the isolated environments were not able to communicate with each other, the process to obtain and monitor system health was achieved by operators logging into each system separately where they collected operational data from each individual cluster. They took this data and aggregated it to determine system-wide health. 

We helped the client replace the process with Prometheus metrics visualized in Grafana dashboards. With a single dashboard, the team can track the health of its applications and can even customize the solution to meet their specific needs. While the solution saved the team from countless hours of manual effort, Amazon Managed Grafana and Amazon Managed Service for Prometheus would even further optimize the team’s productivity, allowing them to forego system maintenance and focus instead on monitoring and collaborative issue management.

As system complexity grows, observability gains in importance in helping enterprises proactively manage system health. With less overhead for the DevOps team, Amazon Managed Grafana and Amazon Managed Services for Prometheus allow AWS users to achieve faster, insights to operate and troubleshoot containerized environments.