Posts
Istio troubleshooting in new scenario that one service to expose into multiple domains and multiple ingress gateways
Since my first time to configure istio with proxy protocol supports in aws, istio and envoyproxy changed a lot. In the past several days, I was exposed to a different scenario that one service to expose into multiple domains.
Things are getting not straighforward, I struggled to make it success yesterday. Today another layer of complexity was added into my cluster: an extra ingressgateway for some services.
For multiple ingress gateways, I followed this article, however my setup is more complex than that.
Posts
Make a S3 bucket public
Turn off "Block public access (bucket settings)" Access control list (ACL): give Everyone (public access) object lists and bucket ACL read. Policy Policy to attach to a role, so the role can be used to upload resources and put the ACL on the uploaded objects.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::demo" }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::demo/*" }, { "Sid": "VisualEditor2", "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::demo-public" }, { "Sid": "VisualEditor3", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject", "s3:PutObjectAcl" ], "Resource": "arn:aws:s3:::demo-public/*" } ] } Uploading command line bash scripts aws s3 sync --acl public-read local-dir/ s3://demo-public/test/ C# source code using (var client = new AmazonS3Client(RegionEndpoint.
Posts
Create another user and grant proper permissions in aws rds mysql
When I tried to create an user for application instead of giving out the master user and its password. I got "Error Code: 1045. Access denied for user 'mysqladmin'@'%' (using password: YES)" when I run "GRANT ALL PRIVILEGES ON demodb.* TO 'appadmin'@'%';".
I tried the method in How do I create another master user for my Amazon RDS DB instance that is running MySQL?, it is for creating another master user with more permissions than an app db user.
Posts
istio virtual service with tls - Connection reset by peer
I got following similar errors when setup my istio clusters.
Mark bundle as not supporting multiuse 301 istio 301 or 404 error:02FFF036:system library:func(4095):Connection reset by peer * Trying 20.190.14.28:443... * TCP_NODELAY set * Connected to kiali.example.com (20.190.14.28) port 443 (#0) * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /home/ng/anaconda3/ssl/cacert.pem CApath: none * TLSv1.3 (OUT), TLS handshake, Client hello (1): * OpenSSL SSL_connect: Connection reset by peer in connection to kiali.
Posts
Reflection on kubernete usage
Today I explained the current infrastructure fleet configurations to our devops team, I have the thought to make a reflection on the things or lessons I learnt in the past and think about what should I do in the next.
The technologies and tools I used in my projects is as following. There are still a lot of things to learn considering the list in my list and the huge ecosystem of kubernetes.
Posts
Resize volumes when PVCs and PVs are okay and the size of file systems in pods doesn't change
Here is an issue with aws-ebs-csi-driver: The size of file system doesn’t change when pvc is expanded. I got the same issue when I tried to do the Curl elk in pods to delete indices this afternoon. I got the message "resize2fs 1.44.5 (15-Dec-2018) open: No such file or directory while opening /dev/nvme1n1" as well when I tried to resize the file system /dev/nvme1n1 in my pod.
As the issue is about csidriver, it is not in the the result of running command "kubectl get csidriver" on my cluster.
Posts
Curl elk in pods to delete indices
Today my staging kibana didn’t show logs. I made the decision to work out a solution to solve issue in hard way this time. I don’t want me in the same situation without solutoins.
When things go wrong, you can’t login kibana to do management or maintainance works. The left option is managing the data from the command line. In the past I figured out to use curl cli in pod to get some information of elk.
Posts
same device mounted on differences mount points
As in my previous article, I gave the following information of my pod. I still have some time before bed, I couldn’t help to seek the reason of that.
/usr/share/nginx/html # df -h Filesystem Size Used Available Use% Mounted on overlay 80.0G 34.5G 45.5G 43% / tmpfs 64.0M 0 64.0M 0% /dev tmpfs 3.7G 0 3.7G 0% /sys/fs/cgroup /dev/nvme0n1p1 80.0G 34.5G 45.5G 43% /dev/termination-log /dev/nvme0n1p1 80.0G 34.5G 45.5G 43% /etc/resolv.
Posts
Resize Pod volumes in eks
I did resize volume of kubernetes in the past, however I encountered an interesting issue when I did the resizing in different way.
According to doc, I should only change the requested size in pvc. Today I changed the size of pv first, then pvc. Here was the interesting thing: all things of pv and pvc are fine, but the size of the file system in pod was not changed.
Posts
Get back my missed keypair of EKS
Today I needed to scale one of my kubernetes clusterss. Those keys are not in my new laptop since I used Mac Air M1. I didn’t see the increase of nodes several minutes after I run the eksctl scale command. I logged into aws console and found there were several "Failed" messages in activity history of the autoscaling group. All the failed messages shown "Launching a new EC2 instance. Status Reason: The aaaa-nodegroup-ng-1-67:8e:b8:8e:33:83:93:68 key pair does not exist.