Archives

Categories

Used Car Quote

Used Car Buying Tips – info on getting your best deal Buying new cars, used car tips and resources for a great deal…

Uncategorized

Introduction to Amazon DevOps Guru & Proof of Concept

0

Introduction
Digitalization and the cloud have made it easier to deploy new features in software applications. This makes error-handling crucial. Any error in the chain, from writing code to deploying to monitoring performance, can immediately degrade customer experience, increase cost or interrupt critical services. Many systems have been designed to manage errors and maintain a chain. However, these systems are limited by the limitations of their tools and systems. IT teams must manually go through terabytes upon terabytes to find the problem. This is tedious and often delays rectification. Companies end up spending valuable resources.
What if there were a quick way to fix operational problems?
The answer is yes, Amazon DevOps Guru.
Amazon has launched DevOps Guru as a fully managed operation service that assists developers and operators in improving the availability and performance of their applications. DevOps Guru allows you to quickly implement improvements to your application by removing the administrative tasks associated in identifying operational problems.
Amazon DevOps Guru in action:
DevOps Guru provides reactive insights that you can use now to improve your application. It provides proactive insights that can help you avoid potential operational problems that could affect your application in future. It uses machine learning to analyze operational data, application metrics, and events to identify patterns that are different from the standard operating pattern. DevOps Guru will notify you when it detects an operational risk or issue. DevOps Guru provides intelligent recommendations to address the current and future operational issues for each issue.
The Proof of Concept:
We have created a use case for AWS to better understand the service. This POC will allow us to dive deep into the service. Click here to see the steps and instructions for performing this POC. Below are the steps and details for performing this POC.
Statement:
We will deploy a CloudFormation Stack, and populate it using test data. This stack will launch a serverless app that includes an API Gateway and DynamoDB Table. To increase traffic, we will change the ReadCapacityUnits of DynamoDB Table from 5 to 1.

Procedure:
Step 1: Select the resource coverage you want and then enable Amazon DevOps Guru Service. After enabling the service, you can also select the resource coverage later.
Step 2: Select the stacks that you wish to monitor with DevOps Gura.
Now, you will need to wait for DevOps Guru’s completion of the resource baselining. This is an important step to establish the expected behavior. We recommend that you wait for 2 hours to complete the next steps for our serverless stack, which has three resources. It can take up 24 hours to complete a baseline when enabled in a production environment depending on the number of resources that are being monitored.
Step 3: Modify your stack and change the ReadCapacityUnits in the DynamoDB Table from 5 to 1.
Step 4: To increase traffic, run the script to trigger API Gateway endpoints in a loop.
Step 5: See the Insights in Amazon DevOps Guru Service to see the recommendations.
Outcome:
API gateway, Lambda and DynamoDB will be affected. The DevOps Guru (DynamoDB) will identify the root cause and provide insights.
POC Screenshots:
These are the images of the POC that were taken.

Fig. 1: You can view various errors regarding the affected resources on the Insights dashboard.

Fig. 2: The Aggregated metrics dashboard shows the timeline of all metrics. You can also compare the origin time for spikes or errors in the metrics. This will help you determine the root cause.

Fig. 3

Ferdinand
Related Posts
Docker Integration with AWS, Azure Clouds Software container firm Docker Inc. announced a beta program to Docker for AWS as well as Docker for Azure. The new offering integrates with the respective infrastructures of the Amazon Web Services Inc. (AWS), and Microsoft Azure cloud. It is described by Docker Inc. as the best way to install Docker and then configure and maintain the deployment. The new initiative includes Docker 1.12 with swarm-mode enabled. The infrastructure integration means users can use a pre-existing SSH key associated with their Infrastructure-as-a-Service (IaaS) account for access control, Docker said, while configuring security groups and virtual networks for simplified Docker setups and operations. The beta program is the same as Docker for Mac or Windows, according to the company.
Uncategorized
Docker Integration with AWS, Azure Clouds Software container firm Docker Inc. announced a beta program to Docker for AWS as well as Docker for Azure. The new offering integrates with the respective infrastructures of the Amazon Web Services Inc. (AWS), and Microsoft Azure cloud. It is described by Docker Inc. as the best way to install Docker and then configure and maintain the deployment. The new initiative includes Docker 1.12 with swarm-mode enabled. The infrastructure integration means users can use a pre-existing SSH key associated with their Infrastructure-as-a-Service (IaaS) account for access control, Docker said, while configuring security groups and virtual networks for simplified Docker setups and operations. The beta program is the same as Docker for Mac or Windows, according to the company.
AWS Brings ‘Intelligent’ Storage to Elastic File System Amazon Web Services (AWS) has added its “Intelligent-Tiering” feature to the Elastic File System service, letting users take better advantage of lower-priced storage classes in EFS. Intelligent-Tiering automates the transfer of items from a regular-priced storage level to a lower-priced tier if there is a change in the frequency that the item is accessed. Regular storage tiers are best for objects that are frequently accessed and used regularly. For objects that are not accessed often or infrequently, lower-priced storage tiers are preferred. Intelligent-Tiering automates the process of moving objects from one tier into another. According to a blog post written by Channy Yun (principal developer advocate at AWS), Intelligent-Tiering was made available to all EFS users on Thursday. He wrote that Amazon EFS Intelligent-Tiering is now available to all EFS users. It automatically optimizes storage costs when data access patterns change. This feature does not require any operational overhead. Intelligent-Tiering will allow objects to be moved across all four levels of EFS storage.
Uncategorized
AWS Brings ‘Intelligent’ Storage to Elastic File System Amazon Web Services (AWS) has added its “Intelligent-Tiering” feature to the Elastic File System service, letting users take better advantage of lower-priced storage classes in EFS. Intelligent-Tiering automates the transfer of items from a regular-priced storage level to a lower-priced tier if there is a change in the frequency that the item is accessed. Regular storage tiers are best for objects that are frequently accessed and used regularly. For objects that are not accessed often or infrequently, lower-priced storage tiers are preferred. Intelligent-Tiering automates the process of moving objects from one tier into another. According to a blog post written by Channy Yun (principal developer advocate at AWS), Intelligent-Tiering was made available to all EFS users on Thursday. He wrote that Amazon EFS Intelligent-Tiering is now available to all EFS users. It automatically optimizes storage costs when data access patterns change. This feature does not require any operational overhead. Intelligent-Tiering will allow objects to be moved across all four levels of EFS storage.
Cyberwarfare: The Origins, Motivations, and What You Can Do In Response Abstract This cybersecurity whitepaper explores the motivations and origins of cyberwarfare and offers suggestions for actions that you can take to counter them. Cyberwarfare: Examples of elements These can be viewed as primarily offensive, but some may also serve a defensive purpose. For a nation to be able perform an act of cyber warfare, these weapons must be developed and maintained. Cyberwarfare tools can be used to attack any nation using many of the concepts that are commonly associated with hacking and penetration testing, such as scanning, system identification, vulnerability scanning, scanning, scanning, and scanning for vulnerabilities. These tools and techniques are used to aid the attacker in gaining greater knowledge about the target’s IT infrastructure. Once the target’s IT infrastructure is known, the next step is to exploit that vulnerability. This is commonly known as gaining access. However, it could also be used to cause damage or destruction rather than remote control or intrusion. Download
Uncategorized
Cyberwarfare: The Origins, Motivations, and What You Can Do In Response Abstract This cybersecurity whitepaper explores the motivations and origins of cyberwarfare and offers suggestions for actions that you can take to counter them. Cyberwarfare: Examples of elements These can be viewed as primarily offensive, but some may also serve a defensive purpose. For a nation to be able perform an act of cyber warfare, these weapons must be developed and maintained. Cyberwarfare tools can be used to attack any nation using many of the concepts that are commonly associated with hacking and penetration testing, such as scanning, system identification, vulnerability scanning, scanning, scanning, and scanning for vulnerabilities. These tools and techniques are used to aid the attacker in gaining greater knowledge about the target’s IT infrastructure. Once the target’s IT infrastructure is known, the next step is to exploit that vulnerability. This is commonly known as gaining access. However, it could also be used to cause damage or destruction rather than remote control or intrusion. Download
CloudWatch Logs is a new AWS feature. This new extension service was introduced at the AWS Summit in New York. CloudWatch used to only monitor resource utilization. To monitor application-level logs, we need to use third-party tools. With CloudWatch Log service, one can upload and monitor various kinds of log files and even filter the logs for particular pattern which could help resolve various production issues like an invalid user trying to login to your application, a 404 page not found error or a bot attempting a denial-of-service-attack. You can also monitor other AWS services such as EBS, EC2, RDS, and others. CloudWatch can store and monitor application logs, system logs as well as webserver logs. These metrics can be set as alarms so that you are notified of any app/webserver issues and can take the necessary actions quickly. CloudWatch Logs: Loggly, Splunk and Logstash already monitor logs and provide detailed reports. CloudWatch Logs is quite basic at the moment, but it would not surprise if Amazon adds additional features in the future. What makes CloudWatch Logs better than other third-party tools? CloudWatch Logs is the only platform that can monitor resource usage and logs. CloudWatch Logs pricing works on a pay-as-you-use model, which can be more affordable than third party tools that use a per node license model. You will pay for log storage and bandwidth to upload files. CloudWatch Logs pricing – $0.50 per gigabyte ingested, $0.03 per gigabyte archived per month Ingested data : This is the data (log file) that CloudWatch is uploading to. Archived Data – All data (log events) uploaded to CloudWatch are retained. You can choose how long you want to keep the data. This data will be archived using gzip Level 6 compression and stored. Storage space for archived data is charged. Let’s review the basic terminologies used in CloudWatch Logs. Log Agent – A python script that directs logs to CloudWatch. Log events: A Log Event is a log file that contains an activity and a timestamp. Log events are only supported in text format. Other kinds of formats will be reported as error in the agent’s log file (located at /var/logs/awslogs.log). Log Stream: A log stream is a collection of log events that are reported by one source. Consider the apache server’s access log. It contains multiple events from one source, i.e. apache web server. Log Group – Log Group is a collection of Log Streams from multiple resources. A WebServerAccessLog, for example, reports Apache access logs from three identical instances. Log Group level is used for Metric filters and retention policy. Metric Filter: The Metric filters tell CloudWatch how to extract metrics from ingested Log events, and turn them into CloudWatch metrics. We can create a Metric filter called “404_Error” that will filter log events to find 404 access issues. To monitor 404 errors on different servers/instances, an alarm can be set up. Retention Policy: The retention policies determine how long events will be retained. Log Groups are assigned policies that can be applied to all Log Streams within the group. You can set the retention time from 1 day to 10 year, or you can choose to have logs never expire. How to install and configure Log Agent on Linux machine: Steps: SSH into the instance and switch to root. Run the following command in the terminal. Shellwget https://s3.amazonaws.com/aws-cloudwatch/downloads/awslogs-agent-setup-v1.0.py1wget https://s3.amazonaws.com/aws-cloudwatch/downloads/awslogs-agent-setup-v1.0.py 3.
Uncategorized
CloudWatch Logs is a new AWS feature. This new extension service was introduced at the AWS Summit in New York. CloudWatch used to only monitor resource utilization. To monitor application-level logs, we need to use third-party tools. With CloudWatch Log service, one can upload and monitor various kinds of log files and even filter the logs for particular pattern which could help resolve various production issues like an invalid user trying to login to your application, a 404 page not found error or a bot attempting a denial-of-service-attack. You can also monitor other AWS services such as EBS, EC2, RDS, and others. CloudWatch can store and monitor application logs, system logs as well as webserver logs. These metrics can be set as alarms so that you are notified of any app/webserver issues and can take the necessary actions quickly. CloudWatch Logs: Loggly, Splunk and Logstash already monitor logs and provide detailed reports. CloudWatch Logs is quite basic at the moment, but it would not surprise if Amazon adds additional features in the future. What makes CloudWatch Logs better than other third-party tools? CloudWatch Logs is the only platform that can monitor resource usage and logs. CloudWatch Logs pricing works on a pay-as-you-use model, which can be more affordable than third party tools that use a per node license model. You will pay for log storage and bandwidth to upload files. CloudWatch Logs pricing – $0.50 per gigabyte ingested, $0.03 per gigabyte archived per month Ingested data : This is the data (log file) that CloudWatch is uploading to. Archived Data – All data (log events) uploaded to CloudWatch are retained. You can choose how long you want to keep the data. This data will be archived using gzip Level 6 compression and stored. Storage space for archived data is charged. Let’s review the basic terminologies used in CloudWatch Logs. Log Agent – A python script that directs logs to CloudWatch. Log events: A Log Event is a log file that contains an activity and a timestamp. Log events are only supported in text format. Other kinds of formats will be reported as error in the agent’s log file (located at /var/logs/awslogs.log). Log Stream: A log stream is a collection of log events that are reported by one source. Consider the apache server’s access log. It contains multiple events from one source, i.e. apache web server. Log Group – Log Group is a collection of Log Streams from multiple resources. A WebServerAccessLog, for example, reports Apache access logs from three identical instances. Log Group level is used for Metric filters and retention policy. Metric Filter: The Metric filters tell CloudWatch how to extract metrics from ingested Log events, and turn them into CloudWatch metrics. We can create a Metric filter called “404_Error” that will filter log events to find 404 access issues. To monitor 404 errors on different servers/instances, an alarm can be set up. Retention Policy: The retention policies determine how long events will be retained. Log Groups are assigned policies that can be applied to all Log Streams within the group. You can set the retention time from 1 day to 10 year, or you can choose to have logs never expire. How to install and configure Log Agent on Linux machine: Steps: SSH into the instance and switch to root. Run the following command in the terminal. Shellwget https://s3.amazonaws.com/aws-cloudwatch/downloads/awslogs-agent-setup-v1.0.py1wget https://s3.amazonaws.com/aws-cloudwatch/downloads/awslogs-agent-setup-v1.0.py 3.
What is a WBS? Bill asked a great question about a WBS and how it can be used to plan project scope. Josh, please clarify. I am only at chapter 7, so if you have any questions, please forgive me. So far, it seems like the WBS is different than the schedule. Is this a point you’re making? Or am I misunderstanding it? I thought the WBS was the basis of the schedule. RegardsBill Bill, you are correct. The WBS is separate from the schedule. You should be able to focus only on the deliverables when creating the WBS. This is true regardless of the time. (I hesitate to recommend phase-based deliverables in general but I recognize that some programs may use this structure) I also recommend that you do not use staff roles to organize your work breakdown structure. Next, you break down the deliverables into tasks. The tasks will be what you use to create your schedule. Your schedule will most likely be influenced at least partially by the WBS structure. But not always. My version of a WBS is called a Product Breakdown Structure (PBS) in the PRINCE2 world. When I speak of task decomposition in a Base of Estimate (BOE), they use the same terminology for what they call a WBS. Although we do the same thing, the terminology can be confusing if you try to translate across the pond. I recommend that you never open a scheduling tool without first completing the necessary steps.
Uncategorized
What is a WBS? Bill asked a great question about a WBS and how it can be used to plan project scope. Josh, please clarify. I am only at chapter 7, so if you have any questions, please forgive me. So far, it seems like the WBS is different than the schedule. Is this a point you’re making? Or am I misunderstanding it? I thought the WBS was the basis of the schedule. RegardsBill Bill, you are correct. The WBS is separate from the schedule. You should be able to focus only on the deliverables when creating the WBS. This is true regardless of the time. (I hesitate to recommend phase-based deliverables in general but I recognize that some programs may use this structure) I also recommend that you do not use staff roles to organize your work breakdown structure. Next, you break down the deliverables into tasks. The tasks will be what you use to create your schedule. Your schedule will most likely be influenced at least partially by the WBS structure. But not always. My version of a WBS is called a Product Breakdown Structure (PBS) in the PRINCE2 world. When I speak of task decomposition in a Base of Estimate (BOE), they use the same terminology for what they call a WBS. Although we do the same thing, the terminology can be confusing if you try to translate across the pond. I recommend that you never open a scheduling tool without first completing the necessary steps.
What is a Project Coordinator? Scott was thinking about the role of Project Coordinator recently. He replied to one my Project Management Career Coaching questions with this: Josh, I am currently attending school for Project Management and have really enjoyed your daily emails. I am currently a Manufacturing Supervisor in a small company. After graduation, my goal is to be a full-fledged Project Manager. I was interested in your recent post about changing industries and how you were successful in landing a job as Project Coordinator. I have been looking for a company for some time, but I don’t have any way to get to know the workers. It is a different industry than mine and I don’t have any connections to the company. I discovered that they have two positions available for Project Coordinators. I am trying decide if it’s worth taking the risk and making the move now rather than after I graduate. It is difficult to gain experience in managing small projects in the position I am currently in. It’s a small company, so I have to accept the fact that I can’t advance in my career unless I leave or wait for someone else to retire. I am asking you this question: Do you think that a job as a Project Coordinator is a good starting place for an aspiring Project Manager. Or would you prefer to stay at the same place for another year and gain more experience in employee management? I know you are busy, so I appreciate any advice you can offer me. Scott, thank you for your email! What is the role of a Project Coordinator? Project coordinator roles are not the same in all organizations. This is why I don’t believe it fits into all strategic career paths. It depends on the industry and the organization, but as a general rule, a role in project coordination is a good way to get into project management. It indicates that the organization is mature in managing projects. It will also give you an opportunity to experience the ‘world of managing projects’ in that organization. If my experience is any indication, you’ll be right there. Building Relationships – I discuss how to get connected to people in the industry or company. If you approach it with respect and tact, you can use LinkedIn to reach out to these people. LinkedIn is a great networking tool that can bring you face-to-face connections and allow you to make friends and develop relationships over time. It’s not a quick process, but it’s possible to expect great results. It will take some time. It won’t take forever, but it won’t stop. It’s ongoing. Some people find themselves in small companies and want to go into a larger company. Once you have the experience and knowledge from a larger company, it is sometimes best to stay there. However, if you want to return to a smaller company, you can bring your expertise and help grow it. If I was you, I would interview for the position of Project Coordinator, even if it wasn’t something I wanted to do. This will give you valuable experience in interviewing and allow you to ask great questions about the company’s project management. It can make a huge difference in your chances of getting hired.
Uncategorized
What is a Project Coordinator? Scott was thinking about the role of Project Coordinator recently. He replied to one my Project Management Career Coaching questions with this: Josh, I am currently attending school for Project Management and have really enjoyed your daily emails. I am currently a Manufacturing Supervisor in a small company. After graduation, my goal is to be a full-fledged Project Manager. I was interested in your recent post about changing industries and how you were successful in landing a job as Project Coordinator. I have been looking for a company for some time, but I don’t have any way to get to know the workers. It is a different industry than mine and I don’t have any connections to the company. I discovered that they have two positions available for Project Coordinators. I am trying decide if it’s worth taking the risk and making the move now rather than after I graduate. It is difficult to gain experience in managing small projects in the position I am currently in. It’s a small company, so I have to accept the fact that I can’t advance in my career unless I leave or wait for someone else to retire. I am asking you this question: Do you think that a job as a Project Coordinator is a good starting place for an aspiring Project Manager. Or would you prefer to stay at the same place for another year and gain more experience in employee management? I know you are busy, so I appreciate any advice you can offer me. Scott, thank you for your email! What is the role of a Project Coordinator? Project coordinator roles are not the same in all organizations. This is why I don’t believe it fits into all strategic career paths. It depends on the industry and the organization, but as a general rule, a role in project coordination is a good way to get into project management. It indicates that the organization is mature in managing projects. It will also give you an opportunity to experience the ‘world of managing projects’ in that organization. If my experience is any indication, you’ll be right there. Building Relationships – I discuss how to get connected to people in the industry or company. If you approach it with respect and tact, you can use LinkedIn to reach out to these people. LinkedIn is a great networking tool that can bring you face-to-face connections and allow you to make friends and develop relationships over time. It’s not a quick process, but it’s possible to expect great results. It will take some time. It won’t take forever, but it won’t stop. It’s ongoing. Some people find themselves in small companies and want to go into a larger company. Once you have the experience and knowledge from a larger company, it is sometimes best to stay there. However, if you want to return to a smaller company, you can bring your expertise and help grow it. If I was you, I would interview for the position of Project Coordinator, even if it wasn’t something I wanted to do. This will give you valuable experience in interviewing and allow you to ask great questions about the company’s project management. It can make a huge difference in your chances of getting hired.
Project Failure: We are at It Again Serendipity happens. I replied to a student Inside pmStudent eLearning with what turned to be an article about project success and failure. Shim wrote, “Projects failure rates?” a few days later. The conventional wisdom is wrong! His blog is amazing. I began leaving comments, and it quickly became one of those comments that should be its own post. Chaos I mean the reports that various organizations have on project failure rates. Shim outlined the following in his post:
Uncategorized
Project Failure: We are at It Again Serendipity happens. I replied to a student Inside pmStudent eLearning with what turned to be an article about project success and failure. Shim wrote, “Projects failure rates?” a few days later. The conventional wisdom is wrong! His blog is amazing. I began leaving comments, and it quickly became one of those comments that should be its own post. Chaos I mean the reports that various organizations have on project failure rates. Shim outlined the following in his post:
Back to top