As next step to proceed further with my AWS experiences I would like to create a data storage, where I can upload some files programatically and retrive an email every time a file was uploaded. Within all this activities some metrics should be generated, so I can see them in CloudWatch service and retrive data with my First Test Application for AWS.
Create data storage
Amazon Simple Storage Service (Amazon S3) is an object storage service and Amazon S3 Glacier is an extremely low-cost storage service, ex. for backup. So I decided to go with Glacier, because I like it cheap for my tests.
First step is to create a Vault, which is a container for storing archives. A Vault is created with a region (EU Frankfurt) and a name (“MyFirstSampleGlacierVault”) and some useful information is shown in creation screen:
Data is stored in S3 Glacier in “archives.” An archive can be any data such as a photo, video, or document. You can upload a single file as an archive or aggregate multiple files into a TAR or ZIP file and upload as one archive.
A single archive can be as large as 40 terabytes. You can store an unlimited number of archives and an unlimited amount of data in S3 Glacier. Each archive is assigned a unique archive ID at the time of creation, and the content of the archive is immutable, meaning that after an archive is created it cannot be updated.
Vaults allow you to organize your archives and set access policies and notification policies.
In the second step I “Enable notifications and create a new SNS topic” and set the topic name to “MyFirstSampleGlacierVaultSNS” in the third step. and I have to “Select the job type(s) you want to trigger your notifications”. As I do not know what this practically means by now, I select both: “Archive Retrieval Job Complete” and “Vault Inventory Retrieval Job Complete”. In the settings of the created Vault I can check, that the Retrieval policies is set to “Free Tier Only”, which is great, becaus it means:
Data retrieval requests that exceed the free tier will not be accepted.
Retrieval Cost: Free
To access programatically to my S3 Glacier Vault I create a new user: “MyFirstSampleGlacierVaultTestUser” with Programmatic access and attach the existing “AmazonGlacierFullAccess” policy directly. As per my current understanding, this allows this user to do everything on every Glacier Vault? I need to check later, if/how I can restrict access to my Test Vault only.
I continue with my test project from my last post.
I have added the entire AWS SDK, I thought. But as I tried to create an AmazonGlacierClient I figured out, that I had to add the Glacier Service SKD to the “entire” AWS SDK:
For my first test I added the credentials to system environment properties and created the /.aws/crendentials file. But this was with credentials for the CloudWatch user. Now I need to use the credentials of my Glacier user.
I found min. three ways to provide the Glacier user credentials.
For the first way I have to add a new section to the credentials file and select this profile:
Lots of deprecated warnings; I’ll ignore them all. Result stays the same, only difference that the access is now denied for Europe:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.amazonaws.util.XpathUtils (file:/C:/Users/i-kau/.m2/repository/com/amazonaws/aws-java-sdk-core/1.11.852/aws-java-sdk-core-1.11.852.jar) to constructor com.sun.org.apache.xpath.internal.XPathContext()
WARNING: Please consider reporting this to the maintainers of com.amazonaws.util.XpathUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
com.amazonaws.services.sqs.model.AmazonSQSException: Access to the resource https://sqs.eu-central-1.amazonaws.com/ is denied. (Service: AmazonSQS; Status Code: 403; Error Code: AccessDenied; Request ID: 1d9d83e6-f301-5137-940d-d42f58994ce4; Proxy: null)
Unfortunatly there is no console (browser) support for Glacier, so I cannot test right now, if this is a problem with the permissions or with the XpathUtils library.
As this is all just for testing, I can live with this error and proceed with testing other services.
I want to start with some practical experiences in AWS, so I go to https://aws.amazon.com, login with my Root user and open the Identity and Access Management (IAM ), where I create aa new IAM user, that I call “MyFirstProgrammaticAccessTestUser”, because the user is of access type Programmatic access. For now, I do not add the user to any group and add only one tag (that I name tag-key) to the user. AWS is warning, that this user has not permissons, but this is fine for now, I will add any permission as soon as the user needs one. Finally I note down the user name, Access key ID and the secret access key.
Set up AWS credentials and region
I am working on a Windows machine, so I create a folder .aws in C:\Users\USERNAME. In this folder I create a file credentials:
To set the default AWS Region I have to create another file in .aws folder: config:
region = eu-central-1
Additionally I have to set this information as environment variables.
I am really not sure, if this is the correct way to set this environment variables, but hey, this is only a test.
I have to go to https://github.com/aws/aws-sdk-java-v2 to get the Clone with HTTPS URL. Then open Eclipse and use the IMPORT dialog to import the project from GIT. After checkout use the Configure -> Convert to Maven project dialog. Then I tried Run as -> Maven install. But this results in a Build Failure:
[ERROR] Failed to execute goal com.github.spotbugs:spotbugs-maven-plugin:3.1.11:spotbugs (spotbugs) on project annotations: Execution spotbugs of goal com.github.spotbugs:spotbugs-maven-plugin:3.1.11:spotbugs failed: java.lang.IllegalArgumentException: Unsupported class file major version 57 -> [Help 1]
I found a clue, that I have to use Java 11 instead of my Java 13. So I downloaded a Java 11 JDK and added it to my Eclipse. But unfortunately I have no clue, how to tell the embedded Eclipse Maven to use this Java 11 instead of Java 13. Great….NOT
Next try: Start a WSL Bash. Need to install Java and Maven first:
Now it took 15 minutes to run until it ends wit an ERROR: There are test failures.
While I was waiting for the WSL-Maven to finish, I figured out, how to tell the Eclipse-Maven to run with the Java 11: I have to create a new Run Configuration where I explicite select the JRE:
The Eclipse-Maven also ends with an ERROR: There are test failures.
But for today I am fine with this result.
Create an AWS Maven Project
I create a new Maven project in Eclipse where I pull in the entire AWS SDK. This is not a good choice for a real world application, where you should only pull in components you need, but for a test project it’s a good start. This is my pom.xml:
020-09-01 19:23:07 [main] DEBUG software.amazon.awssdk.request:84 - Sending Request: DefaultSdkHttpFullRequest(httpMethod=POST, protocol=https, host=monitoring.eu-central-1.amazonaws.com, encodedPath=, headers=[amz-sdk-invocation-id, Content-Length, Content-Type, User-Agent], queryParameters=)
2020-09-01 19:23:08 [main] DEBUG software.amazon.awssdk.request:84 - Received error response: software.amazon.awssdk.services.cloudwatch.model.CloudWatchException: User: arn:aws:iam::175335015168:user/MyFirstProgrammaticAccessTestUser is not authorized to perform: cloudwatch:ListMetrics (Service: CloudWatch, Status Code: 403, Request ID: 75f02535-28c7-49c8-930a-b8d8449c625a, Extended Request ID: null)
Exception in thread "main" software.amazon.awssdk.services.cloudwatch.model.CloudWatchException: User: arn:aws:iam::175335015168:user/MyFirstProgrammaticAccessTestUser is not authorized to perform: cloudwatch:ListMetrics (Service: CloudWatch, Status Code: 403, Request ID: 75f02535-28c7-49c8-930a-b8d8449c625a, Extended Request ID: null)
So the Error is:
user/MyFirstProgrammaticAccessTestUser is not authorized to perform: cloudwatch:ListMetrics
I try to solve this by going back to the IAM console and add the user to a new created group with attached policy “CloudWatchFullAccess”.
So this worked, this was quite intuitive 🙂 The result is empty, I guess because of the metric-namespace that I initaly set with placeholder name. I looked into my CloudWatch Dashboard, but could not find any metric with data. I guess, I have to create a metric and find a way to create data for the metric. TBC
Today I made my first AWS Certificate and I want to share this.
I have to login to AWS Certification with my APN account. There I had to create a ne CertMetrics account on the first visit, that automatically got connected with my APN account. Afterwards I can open the CertMetrics page, where I had to enter some information about me first. Then I could go to the Digigal Badges section, where I had to sign in to another Platform: Credly’s Acclaim Platform, to create another new account. On my Credly’s page I cannot see any badges or certificates.
Now I have two additional accounts, CertMetrics and Credly’s, but still can not share my certificate. Maybe it is not possible to share this certicate, maby I have to do the AWS Cloud Practicioner certificate first to share something?
In my second Blogpost I wrote about a bug, that was not a bug, but then I had to figure out, that there is a bug.
The syntax hightlighting is working while editing a blogpost or viewing it as single post. But not on the home page. To enable it on the home page I had to change the ‘Enable Code Block on Home Page’ setting. Works fine.
I wrote my second Blogpost about this and then the syntax highlighting was gone again.
I disabled the second post: The syntax highlighting of the fist post was there again. I enabled the second post again: The syntax highlighting of the fist post was gone :'(
Next test: I added block of code in my second post. The code of the second post was highlighted on the Home Page. And also the code of the first post on the Home Page!
I removed the block of code from my second post and added this cool piece of code to this post:
10 PRINT "Hello World"
20 GOTO 10
Now the syntax hightlighting works on home page for this (third) post and the first post. As long as the first post on the home pages has a code block, the code blocks of all posts on the home page will work.
So if you see an post on my home page without syntax highlighting, just wait until I write a new post with a piece of code or open the single post page.
I made my first Blogpost and I found a bug, that the code snipet was highlighted while I was editing the post or in preview mode. But when I open it for example on my mobile or in private mode or just in the same browser in “normal” view, the syntax hightlighting was gone.
This is how it looks in edit or preview mode:
And this is how it looks in “normal” mode:
I thought, this would be a bug, but at the end it turned out, that this is a feature.
I contacted the Plugin Author through the support forum and this is his answer:
I followed his instructions and now the sourcecode shines highlighted as expected!