REM download https://github.com/containers/podman/releases/download/v5.2.5/podman-remote-release-windows_amd64.zip REM unzip it to C:\DIM\podman-remote-release-windows_amd64 SET PODMAN_HOME=C:\DIM\podman-remote-release-windows_amd64\podman-5.2.5 set PATH=%PATH%;%PODMAN_HOME%\usr\bin podman machine init podman machine start set DOTNET_ASPIRE_CONTAINER_RUNTIME=podman Get podman desktop from https://podman.io/

A newly created postgresql container will do a following steps. They are quite time consuming in integration testing and add some unpredictable factor. If a integration testing will clean up tables and data, volumes can be used to speed up postgresql container. pg_ctl -D /var/lib/postgresql/data -l logfile start initdb stop postgresql and start again There might be one issue, do I need to maintain the same credential?

Since wrote Things learnt from recent preparation of 1T data for testing, I had figured out how to do that on the following weekends. Today I couldn’t get the steps to do so immediately. It is better to note down for my next time usage. The detailed steps is as following Find the location in the file Get the block contains contents to modify Extract the block to a file and modify it

Several things learnt from recent preparation of data for testing. Question the way when it takes too long to process data sed: remove last n lines In the past I always thought sed is the fastest way to manipulate data. Is it true for all the cases? I generated 1T testing data in ndjson format, however the last lines of those files are corrupted json data. I had to remove the last line.

One thing constantly bothered me since I used mermaid for flowchart, how to correctly group elements in a group. Only today I have a thought about what will happen if I put node in subgraph without links between them. Finally I got it works to correctly group elements in a group. Another things, mermaid diagram is natively supported in asciidoctor now. This one is my first example to use it.

In Beating the average, Paul Graham mentioned lisp is the secret weapon behind his startups, he also elaborated on that point. I believed deeply that after I wrote some small programmings in common lisp before I learnt reddit pilot from lisp to python. Since then I always doubt if the power of lisp is still relevant considering that java, dotnet, python and ruby ecosystems are more mature than that time.

Due to security policy, I had to set dotnet_cli_home to non-default location to run some building steps. Things went quite well until not. All my command line usages and batch scripts work, until I tried to create a project using the custom project template via Visual Studio and noticed that old deleted template was shown in "Create a new project" dialog. I tried several options and none of them works for my case.

Most time, I think json is forward and backward compatible as long as some convention is kept such as adding fields only etc. I always thought so since I studied CQRS and event sourcing. Is it true for FHIR, especially firely sdk? I answered so in a discussion, however I doubted it as I didn’t do it in FHIR. Here I have a short code to demostrate the two case. In the end, the result is yes and no.

uninstall vs and vsinstaller restart machine download below two vs cpp redist and install first before install visual studio https://aka.ms/vs/17/release/vc_redist.x86.exe https://aka.ms/vs/17/release/vc_redist.x64.exe download the latest version of VisualStudioSetup.exe from https://c2rsetup.officeapps.live.com/c2r/downloadVS.aspx?sku=professional&channel=Release&version=VS2022&source=VSLandingPage&cid=2030:a4e034a4e73f4c1cb41366109a953e4d choose download all and install when installing visual studio after choosing components to install Clear Visual Studio Component Cache Close Visual Studio (ensure devenv.exe is not present in the Task Manager)

In functional programming languages, there are several difference betwee them. I was quite interested in functional programming, even I spent some time to learn haskel and scala. I knew what I can use Select for map, not sure about folder and reduce FeatureFoldReduceInitial ValueRequires an initial value.Does not require an initial value.Function Signature(A, B) => A(A, A) => AApplicabilityCan be used on empty collections.Requires a non-empty collection.VariantsfoldLeft and foldRightreduceLeft and reduceRightStarting PointfoldLeft starts from the left.

Due to the nature that it is difficult to debug msbuild custom task, I thought if I can use it in a way to streamline my debug workflow. one pack command will complete the following steps. - pack - add the package into local feed source - add the package to global-packages folder My current workflow is below. It is a manual process. It is realy a pain this way: - pack - nuget add [package-name] -source C:\nexus\.

Most information about custom task of msbuild are quite basic, it will get complicated when it is involved reflection. I tried several ways according searching results and answers from chatgpt: reflection, assemblyLoadContext and mono cecil. The most promising one is mono.cecil. I tried directly loading assemblies and via AssemblyLoadContext. They all failed to load some microsoft assemblies. It looks awkward even I can change my build scripts to copy those assemblies.

I shared one of CICD I designed and implemented in linkedin. It is good to keep here as well. I researched several options and put my thought at Reflection on implementation of CICD using argo-workflows Nowadays, argo-workflows is quite mature. I will use argo-workflows for CI part in my future projects. It is definitely worth to invest one’s time on argo-workflow as it is widely used.

What will you do when you encountered a runtime version is disallowed for specific version say 8.0.6 in building environment? Install the specific runtime version, right? Please hold, I show you my story. I initially thought I couldn’t run the below commands to install it, it is not. After several attempts, My colleague spotted the difference between runtime version and sdk version. They are different. A runtime version is supported by several sdk versions.

Nuget local feed Prerequisites nuget.exe is installed in C:\packages and in the command search path. Setup local feed Assume C:\packages\.nuget-local\packages will be the locations to install local packages. and several local dev packages need to install. Packages below: - "C:\packages\test.4.1.0-beta-24159.1.nupkg" - "C:\packages\testa.4.1.0-beta-24159.1.nupkg" - "C:\packages\testb.S3.4.1.0-beta-24159.1.nupkg" Run below commands to install above packages into source C:\packages\.nuget-local\packages REM run command line with C:\packages as working directory nuget.exe add tes.

Just an architecture style MACH got me attention. It is short for micro service based, API first, cloud native, headless. There are two of projects I involved one are in this architecture style. One is sitecore based, another one is developed. Kubernetes, nodejs, npm, strapi tsoa are used. Their mapping is as following. micro services based: multiple services based on expressjs api first: tsao cloud native: more advanced than cloud native, kubernate native is used.

In the past several months, I called it quite intensive. I need to master the knowledge of FHIR, advanced dotnet technologies (attributes, generator, reflection, code emit, expression tree, advanced linq, customized aspnet pipeline, nuget package), code optimization technologies and several aws technologies (DynamoDB, S3, glue, spark) in depth to meet the schedule of a project. You really can’t account on solution architects or support engineers from cloud providers. I don’t get one perfect answers from them so far when I used their services.

It is about three months now. I make reflections on things I have done well and bad. I will refine my learning workflow. In the past, I used 2nd-brain, daily-logging, flashcard and spaced-repetition systems to help to accelerate my learning under the circumstance that I can’t get fully complete information from different stake holders. I was left with only way to find it out by myself. I read all material I thought will be useful and noted them down in my logseq vault.

I am passionate about technological details and how things work under the hood. Recently I didn’t get confirmative answers from AWS. I spent a half day to re-read the famous paper. and the new one Amazon DynamoDB: A Scalable, Predictably Performant, and Fully Managed NoSQL Database Service. I think the following several concepts can be helpful to under its predictable performance. Consistent Hashing and zero hop DHT Storage node and 10G partition Typical flow of putItem I also put all of the three pictures into a slides and put refereneres at the last slides.

I always thought SaaS are based shardig, load balance and multiple-tenant. In the past several days, I re-read Dynamo and DynamoDB papers to figure out some key info to share with my colleages to find a common ground about DynamoDB. One thing is mentioned many times in DynamoDB is token-bucket. Suddenly a thought popped out my mind when I walked back after lunch with my colleague that token-bucket algorithm should be a component of SaaS and multiple-tenant.

I used mindmap in my logseq, however it show left to right side, the start node is always at the left side. I prefer to have the main concept shown in the center. Jsmind is the perfect fit after I saw its demos. Here is I create a shortcode in my hugo and try to create a demo mindmap using jsmind. "\n{\n \"meta\":{\n \"name\":\"jsMind remote\",\n \"author\":\"hizzgdev@163.com\",\n \"version\":\"0.2\"\n },\n \"

Best practice or "aws configure" will put ~/.aws/config and ~/.aws/credentials in the 600 mode. however if you follow Developing using a Docker image, you will get several errors e.g. cant' find region, reach endpoints, or no access to resources. I run docker command as a normal user, docker daemon run as root user. The below scripts will run a docker image which default user is glue_user (10000). the mounted volume ~/.

I tried to use tikz in my logseq. However it fails to render correctly. Today I have time to try it in my hugo sites. Here is a demo from https://tikzjax.com/ \begin{tikzpicture} \draw (0,0) circle (1in); \end{tikzpicture} )

When I was preparing a slides to recommend broad usage of JsonSchema, somethings I thought are good to have for general public usage. Those things troubled me for a while during I authored a JsonSchema. Versioning Currently there is no standard way how to define a version for a schema. I might use a version property for that. SemVer is good fit for the version content schema. Use if and else for different version.

Microsoft FHIR Server serialize searchindices this way in cosmosdb. How to model it in AWS DynamoDB? { "id": "9240cda7-bbb7-4ad4-aa9f-971750a3dc71", "searchIndices": [ { "p": "_id", "c": "9240cda7-bbb7-4ad4-aa9f-971750a3dc71" }, { "p": "_lastUpdated", "st": "2018-08-22T23:37:56.1289012+00:00", "et": "2018-08-22T23:37:56.1289012+00:00" }, { "p": "patient", "r": "Patient/example" }, { "p": "performer", "r": "Practitioner/example" }, { "p": "related", "r_0": "Observation/example-genetics-1", "s_1": "http://hl7.org/fhir/observation-relationshiptypes", "c_1": "derived-from" }, { "p": "related-target", "r": "Observation/example-genetics-1" }, { "p": "related-type", "s": "http://hl7.org/fhir/observation-relationshiptypes", "c": "derived-from" }, { "p": "status", "s": "http://hl7.

Background I worked on APIs and OpenAPI Spec (OAS) in past projects, even I used stoplight and NLP for open-source API style guide enforce and linting. I thought I was quite good at jsonschema. I dont’k think so after recent usage of jsonschema for a very aspnet flexible configuration file. Configurations in dotnet are quite straight forwards if all the configurations following the Options pattern. ConfigurationProcessor is not that case.

Yesterday, I clicked a button in one web page, no response from that. I thought it must be something wrong. I opened browser console and noticed there is an icon. It turned out be a copilot icon after reading the logs in console. Nowadays, copilot almost is everywhere, office suite, vscode, visual studio, windows and edge browser. Never thought it is embedded in browse web console.

Prologue One or two years ago, I tried several times to install kind cluster to using cilium cni. However I didn’t make it. Today I really want to setup one after reading an Kind cluster with Cilium and no kube-proxy and considering that major kubernetes distributions are using cilium cni now. After about 3 hours, I finally got it running successfully. Things don’t go smoothly. Here are my steps to setup it up.

Stackoverflow give several ways to resolve localhost HSTS issue (redirect from http to https). My laptop is managed by my employer and registry can’t be edited. "How can I see which sites have set the HSTS flag in my edge browser?" in chatgpt give me an answers. One step of the answer get my attention. I can change HSTS in my application for development environment. It is too late now.

I searched web, first several links in SERP are not the offical one from microsoft. I tried them and failed to install mssql due to errors Error: 'deb [arch=amd64 https://download.docker.com/linux/ubuntu \ xenial \ stable' invalid] or Can’t install MS SQL Server on Ubuntu 21.10. At last I follow the instruction from microsoft. I successfully installed mssql server in my wsl+ubuntu22.04 environment. curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | sudo gpg --dearmor -o /usr/share/keyrings/microsoft-prod.

I used .netrc in several projects and I wrote something about that. I thought it is a convenient way for git until recently. Recently I have a new laptop and want to setup my git credentials. This time, I noticed .netrc might not be secure as other applications that use cURL to make requests to servers defined in your .netrc file will also now be authenticated using these credentials.

How do you extract multiple fields at different levels of nesting json resources? jsonpath, jq, jmespath or chatgpt? In FHIR, bundle resource in json format usually have multiple level of details nested in json. It’s quite different from other use cases. My first thought is using jsonpath as it is quite similar to FHIRPath. There are several limitations of jsonpath which make it infeasible. It doesn’t support to the use case, even though JsonPath supports deep scan and Bracket-notated child or children.

Standard REST API has limited verbs if using http verbs. However in reality and many business contexts, they are not enough. In the past, many options are proposed. Today I found another option in FHIR. It might be helful to put them here for comparison. Control data in request In stackoverflow What to do when you need more verbs in REST, cheeso gave the answer below. It is like the way json-rpc, not well-specified version.

I learnt to extract npm package inform from https://registry.npmjs.org/ on 2023-08-31. Today I have more packages. The original query need to be optimized for better performance. In the old post, one can quickly spot that the package information is fetched twice. It is the place to optimize today. I will put selected fields into a column, and extract values from the column later. This way, It can spend less time.

Today I spent about 1 hours in the task Deploy Lamp Stack on Kubernetes Cluster - Task. This task deepen my understanding of ini file handling and volume mounting with subpath. I think there are 2 tricky parts make the task interesting. First, the similarity between --from-literal=variables_order=EGPCS and variables_order = "EGPCS" in php.ini make one try to create a configmap at first. However it is not the case. the actual volumemount will mount each key as a file

Today I got certified as Certified Kubernetes Security Specialist (CKS). It marks a milestone in my mastery of kubernetes after a long journey. It is a good time to reflect on lessons what I learnt. My experiences with kubernetes from 2018. I have a habit of comparing other optiosn before making my decisions. Before 2018, I used vagrant and other configuration management tools (puppet, chef, ansible) to setup my lab environments.

Seperate view dll issues Put views (cshtml) into the main web project 'RazorTargetAssemblyInfo.cs' could not be found GenerateAssemblyVersionAttribute is set to true : af file MyProject.RazorTargetAssemblyInfo.cs is generated <Project Sdk="Microsoft.NET.Sdk.Razor"> <PropertyGroup> <TargetFramework>netcoreapp3.1</TargetFramework> <GenerateAssemblyVersionAttribute>true</GenerateAssemblyVersionAttribute> </PropertyGroup> https://github.com/dotnet/aspnetcore/issues/37795#issue-1034241118 The namespace 'Razor' already contains a definition for 'Template' https://stackoverflow.com/a/58140724/1101691 Duplicate 'Microsoft.AspNetCore.Razor.Hosting.RazorLanguageVersionAttribute' attribute How to decompile/extract(cshtml--(razor view)) from dll https://stackoverflow.com/a/69951824/1101691

I spent almost a whole afternoon today to restore a project collections . It doesn’t go well. Azure devops server might supports SQL Server express version. and I choose the wrong way to do that. After several trial, I figure out the way to restore a project collections of azure devops to another instance. The steps is as following. install sql server (not sure if sql server express is compatible)

I used dotpeek for a while to decompile dotnet assembly. Only yesterday I finally found a way to export project from dotnet assembly. The picture below is the menu to do that in dotpeek. However the version I used doesn’t perfectly decompile lambda expression. The decompiled code of lambda expressions are very hard to understand. Ilspy is one of decompiler that can decompile lambda in a understandble way.

The fastest way to get nodes Many times, I run "kubectl get nodes" to get nodes I need to connect to make changes. In the question description section of tasks, master nodes and worker nodes are given. I think the fastest way to get nodes and I wasted some time on getting that info from commands. Get events Now kubectl has events command. The events command has intuitive options and less keystokes.

Clicking the copy button of plantuml plugins of vscode, one can get png from plantuml files. however png files look blurry when even slightly resized. Is there any better way to avoid that ? SVG is a good option. Plantuml provides the option to export as svg. It might be troublesome to get a svg file by several steps using it. It is also slow. Is there any other better options?

My credit of Visual Studio Professional Subscription azure is $50. In the past two days, I activated my subscription. I didn’t go IaC way to setup my POC environment as I did’t have enough time to do that. After I setup more services, the 10 credit is used. It can be quickly used up if resources are kept running. I know there is a way to import existing resources as I used terraform in the past.

I daily use anki to review things. However I have issues to run it in wsl2+ubuntu 22.04 environment. The only solution for me is below. The workable version is 2.1.66 Qt5 with the following environment variable export QTWEBENGINE_CHROMIUM_FLAGS="--disable-seccomp-filter-sandbox"

After thrice attempts, I finally successfully completed the task Selinux Installation at https://engineer.kodekloud.com/practice. Since it is not straightforward, I posted here. > sudo yum install -y selinux* > # SELINUX=disabled > # change SELINUX in /etc/selinux/config > cat /etc/selinux/config | grep SELINUX # SELINUX= can take one of these three values: SELINUX=disabled # SELINUXTYPE= can take one of three values: SELINUXTYPE=targeted

Say in a situtation where budget is fixed, what will you do to improve system performance? I came out several solutions to this at first, later I came out with more thoughts. I list them here for later reference. create selective indexes partition tables materialized view asynchrous processing: put non real-time operations as a job queue, spread jobs to non business hours, non peak loading time etc.

In stripe doc, Stripe explain how it works as following quoted contet. Other places only mentions that the processes will be only once. Stripe's idempotency works by saving the resulting status code and body of the first request made for any given idempotency key, regardless of whether it succeeded or failed. Subsequent requests with the same key return the same result, including 500 errors. Then there is an interesting question how to handle timeout and retry cases?

I leant API idempotency many years ago when I worked in Asia Fusion Technologies. At that time, I was involved in integration with payment systems and games from different vendor. That gave me a chance to read a lot of API documents and evalute them. During that time, one interesting characteristic of API got my attention. It is API idempotency. That time, there is no much information of idempotency how to implement etc.

Today I read agones site and noticed it play asciinema cast in its home page. I wondered if I can put my asciinema cast into my blog. Here is my one. AsciinemaPlayer.create('/casts/497308.cast', document.getElementById('497308'), { }); The cast is about how to use titl for local micro-service development based on kubernetes.

My first attemp to resolve "Kubernetes Time Check Pod" in https://engineer.kodekloud.com is not successful. The exercise doesn’t allow to open an editor to edit a yaml file. I had to resort to cat. I attempted several times to resolve it on my local cluster, still no luck even I tried different combination such as command, args,string in string, avoiding expansion. This one from stackoverfow gave me a clue how to resolve my issue.

In the past several days, I practiced several exercises for my coming CKS exam. I noticed that it takes a while to get static pods restart after configuraiton changes. I was curious if there is any faster way to make that happen. I think the following ways definitely will be faster than waiting 3 minutes after I used them for a while. The observable time is shorter than 3 minutes

I encountered an issue how to get detailed information based on statistics information such as minimum and maximum values in a dataset. The traditional way might be use case expression,coalese functions etc to archive the goals, However I felt it is quite clumsy. How do it in a neat way? In SQL standard, there is a way using unpivot to transform columns to rows. It is a good fit for this situation to get a neat SQL statement to resolve the issue.

I encountered the issue yesterday during my exercise in killer CKA similator. However I made the same mistakes again. I think it worth to write it down to avoid I make the same mistake again. In my day to day works, I usually add volumes before containers in the spec section of pods. In exams, sometimes I need to add extra volumes into a existing pod extracted from a running pod, and I still add volumes in the old way.

Beyond High Availability: The Importance of FMEA in Ensuring Uninterrupted Operations Introduction In an increasingly interconnected and data-driven world, high availability is a critical component for any system or infrastructure. High availability ensures that a system can continue to operate and deliver its services, even in the face of hardware failures, software glitches, or other unforeseen disruptions. However, high availability alone is not enough to guarantee uninterrupted operations. To achieve true resilience, it’s essential to complement high availability with a proactive approach to risk assessment and mitigation.

Do you know how to skip first line before data pipe into xargs? Today I need to export custom resources and want to avoid error messages because the column names line in the output of kubectl. I knew there is a command line option (--no-headers) of kubectl and managed to get the option, but I don’t want to spend time to search the option in future. Is there any way to skip first line before data pipe into xargs?

Nowadays, zero downtime is normal, you can’t believe that in azure app service changes in "app settings", App Service triggers an app restart. The alternative ways to do avoid that is to use sentinel key with App Configuration. however App Configuration will cost a little. Do we really need to use App Configuration to achieve that ? The naive and obvious way is that changing appsettings.json files via entering the app services using kudo console.

Last week I worked on filling info package information to a workbooks. I used VBA macro before and thought of it again to save some of my time. Things is changing very fast, the new javascript based custom fuction is the mainstream. The initial one I followed the tutorial, however I didn’t find any way to add it into "My Addin" as described in microsoft doc site. I don’t have administrator rights on my laptop.

Background I didn’t find a rule implementation of spectral at stoplightio and github to lint the rule that resources should be nouns in REST API. stoplight mentions that Use Nouns for Resource Names, however there is no another article about that. I was bothered by that for a while to the point I want to give it a try to implement my rule for that. There are several existing implementations about this rule.

Today I tried to integrate identity into mvc projects. The main doc about this is at https://learn.microsoft.com/en-us/aspnet/core/security/authentication/scaffold-identity?view=aspnetcore-7.0&tabs=netcore-cli#scaffold-identity-into-an-mvc-project-without-existing-authorization dotnet add package Microsoft.VisualStudio.Web.CodeGeneration.Design dotnet add package Microsoft.EntityFrameworkCore.Design dotnet add package Microsoft.AspNetCore.Identity.EntityFrameworkCore dotnet add package Microsoft.AspNetCore.Identity.UI dotnet add package Microsoft.EntityFrameworkCore.Tools # add second dbcontext for identity and authorinization, use database sqlite dotnet aspnet-codegenerator identity -dc MyApplication.Data.ApplicationDbContext \ --files "Account.Register;Account.Login" --databaseProvider sqlite dotnet ef migrations add CreateIdentitySchema \ --context MyApplication.Data.ApplicationDbContext dotnet ef database update \ --context MyApplication.

Background I was once asked how to run big scaled kubernetes clusters regarding etcd size limitation. For example how to handle 8G limitations. I was not satisfied with my answers such as increasing memory, compaction & defragmentation as I knew there were several big scale clusters in several companies and 8GB might be a hard limit. However I didn’t have any clues how they make that happened. Since then, I kept tabs on etcd by reading etcd community meeting (Public).

I was very excited to be invited to give a talk about dapr. Below is my talk. My demo in the session is not successful as things in kubernetes and dapr field are changing very fast. My lab environment didn’t work as expected. Finially I make it work today, I share it here below. Here is my talk at 18th July from time 2235 in below video. Prerequisites kubectl, kind, docker, dapr, git, vscode in WSL

A typical steps involved in the issuer response process for a card authorization, a simplified description is as following. Receive Authorization Request: The card issuer system receives the authorization request from the merchant or payment processor. Card Validation: The issuer system validates the card details provided in the request, including the card number, expiration date, and CVV/CVC. Account Verification: The issuer system verifies the cardholder’s account details, such as account status, available funds, and credit limit (if applicable).

charge backSend Authorization RequestMerchant HandlingReceive Authorization RequestCard ValidationAccount VerificationFraud DetectionDecision MakingGenerate Authorization ResponseSend Authorization Responsemerchant/payment processorcard issuer system

I think I am a big fan of textual UML after I learnt it from thoughtworks techradar. I doubted it after my recent experience. In the past 2 months I spent long periods of time to think of how to make my Powerpoint slides pass my boss Lim Chee Kiam's standard. I thought deeply about that. One moment, I asked myself why Grady Booch, Ivar Jacobson, and James Rumbaugh didn’t come out a diagram which cater to dynamic(behaviour) and static (structure) diagrams, then I realized it will be a mess if those two things are put toghere.

I chose logseq as my PKM I was overloaded with information in the past several weeks. I sought many ways to overcome it. One choice I made is using logseq as PKM. why? There are several open source or freeware verion of PKM. Why did I choose logseq? logseq is clojure based. I am a big fan of lisp-liked languages. I studied and did some coding using racket, common-lisp, elisp and clojure after I read Paul Graham’s essays about lisp.

Installation I installed Ubuntu 22.04 on my laptop following the scripts here. sudo apt-get update sudo apt-get install ca-certificates curl gnupg sudo install -m 0755 -d /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg sudo chmod a+r /etc/apt/keyrings/docker.gpg echo \ "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin service docker start Troubleshootig It is little different from my previous installation.

Item Hash/encryption UUID Order preservation possible (linear hash) possible ( customized UUID algorithm) length shorter or longer fixed ubiquitous anyone or machines can be used to hash depends performance (computing) ? ? performance (search) fast slow References:

Last week I was assigned a task to keep search result pages of a site of my company up to date. After researching, trial and errors, I found out the following ways to keep the search result pages in search engines up to date. Remove outdated content from search result of google and Bing. temporary: block URLs from webmaster tools permanent: get sitemaps.

I never thought I will learn to use powershell seriously. Now I joined NTUC, there are many restrictions. Today I started to use linux utils equivalent in powershell. dig: Resolve-DnsName telnet: Test-NetConnection host: Get-Host touch: New-Item cat: Get-Content tail: Get-Content grep: Where-Object or Select-String man: Get-Help References https://mathieubuisson.github.io/powershell-linux-bash/#pwd- -

2 days ago, I didn’t figure out how to filter out data using power-query. I used a workaround solution to achive the same goal. Yesteray I occasionally found that I can filter out data directly without using my workaround solutions. Table.SelectRows Table.SelectRows( Table.FromRecords({ [CustomerID = 1, Name = "Bob", Phone = "123-4567"], [CustomerID = 2, Name = "Jim", Phone = "987-6543"], [CustomerID = 3, Name = "Paul", Phone = "543-7890"], [CustomerID = 4, Name = "Ringo", Phone = "232-1550"] }), each ( [CustomerID] > 2 and Text.

In my previous two companies, everytime I used the subdomain way without thinking much about this. Recently I wondered the rational behind this decisions. I had the clear answers two days ago. Today I have the time to put it into this writing. Factors subdomain(dev.example.com) subdomain reasons path (example.com/dev/) path reasons server side performance 9 dns records point to differnent IP.

Leader election in Kubernetes control plane - #HeptioProTip give a way how to find out leaders of Kubernetes control plane components kube-scheduler and kube-controller-manager. However it was based on a old version of kubernetes, and the mechanism is changed in later versions. Today I was asked how to find the leaders. Here is the new way to find. $ kubectl -n kube-system get lease NAME HOLDER AGE kube-apiserver-c4vwjftbvpc5os2vvzle4qg27a kube-apiserver-c4vwjftbvpc5os2vvzle4qg27a_b187371d-e48c-4216-8228-707a0ecf6100 2m57s kube-apiserver-dz2dqprdpsgnm756t5rnov7yka kube-apiserver-dz2dqprdpsgnm756t5rnov7yka_0b531f66-0c31-453c-9277-a6c1aa81da94 86s kube-apiserver-fyloo45sdenffw2ugwaz3likua kube-apiserver-fyloo45sdenffw2ugwaz3likua_3e322f9a-9724-4e3a-9fc6-a512e9424164 2m11s kube-controller-manager kind-control-plane_bec39b96-87c4-4bce-8775-2eeb4eb4c1e8 2m53s kube-scheduler kind-control-plane_db6f36d8-ceaa-40eb-b821-75f8ae829f22 2m53s $ kubectl -n kube-system get pods -l component=kube-controller-manager,tier=control-plane \ -o custom-columns=NAME:.

Progressive delivery: Istio Canary Deployments using flagger. Steps environment: kindest/node:v1.25.3 Istio 1.17.0 helm # need to install metrics-server, HPA depends on metrics-server kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml # command: # - /metrics-server # - --kubelet-insecure-tls istioctl manifest install --set profile=default kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.17/samples/addons/prometheus.yaml helm repo add flagger https://flagger.app kubectl apply -f https://raw.githubusercontent.com/fluxcd/flagger/main/artifacts/flagger/crd.yaml helm upgrade -i flagger flagger/flagger \ --namespace=istio-system \ --set crd.

Last week and this week, I completed my basic learning of APMs as a tool of SRE. I learnt fundamentals of datadog and newrelic, walked through their tutorials in last week and this week. I don’t find a easy way to learn dynatrace from its site. The whole learning experiences between datadog and newrelic are on par with each other. What’s the deal breaker when making choices among them? Leader Feature flags intergration During a discussion about progressive delivery using flagger, I was curious if it is possible to integrate feature flags and APM.

It is a big topic to setup kubernetes correctly. I will keep updating this when I find something new or missed in this.

The explanation of pod sandbox at the abstraction that replaces the "pause" container that is used to keep namespaces open in every Kubernetes pod today. I doubted about that. What’s the point to introduce a new concept? Yerterday I went down the rabbit hole to understand it. This morning I finally got the hang of it. Kubernetes blog said it is an environment. It maybe a VM, a group of containers.

In the past I learnt a lot of kubernetes, however there is no much information about kubelet or not deep enough. I couldn’t answer the questions about kubelet correctly in an interview. Here I put all the information I collected from the internet here for my references. kubelet component architecture how does kubelet work Container Lifecycle Management Through the CRI kubelet, CRI and CNI sequence diagram kubelet, CRI and CNI interaction diagram The process of creating a pod Handler The work of podWorkers syncPod kubelet Runtime Creating a sandbox for a pod References: https://www.

Components Description Metrics Scaler Metrics Server Metrics Server collects resource metrics from Kubelets and exposes them in Kubernetes apiserver through Metrics API for use by Horizontal Pod Autoscaler and Vertical Pod Autoscaler. Metrics API can also be accessed by kubectl top, making it easier to debug autoscaling pipelines. Metrics Server is not meant for non-autoscaling purposes CPU/Memory

Usually I explain one of my CICD pipeline to others with words such like github action + argo-cd. I am not sure if they understand it. A picture is worth a thousand words. Here is my CICD pipeline picture. I used excalidraw to draw the diagram. Excalidraw link is https://excalidraw.com/#json=zJHTTZ_QgcarkYCCankS8,gotb3rzXtE8ux810NoK18A. Feel free to use it.

Usually I tell others that I have about 5 years experience of aws services. Today I want to know more about how many years experience I have. I logged in my aws account and tried to find my account creation date. There is no such information in account page. I was so eager to find the information and searched for alternative way to get that information. The easiest way is good one, however I already deleted the email.

AWS has almost established themselves as the de facto public cloud provider for a lot of businesses, other providers always try to provide similiar services and similar APIs to attract existing AWS customers to migrate to their platform. Each following provider provides similar comparison table comparing to AWS. Those comparison can give one a quick start to adopt their services without major issues. Even those following providers provide migration tools for migrating from aws to their platform.

Summary: Reduce the chance of resource conflict in gitops by reducing the time window of using shared resource files My initial journal with gitops was setup my gitops cicd pipeline following the flux tutorial way. I immediately found out the limitation of flux at that time and sought alternative solutions. After comparing different solutions, I settled down argo-cd. After a short while usage of argo-cd, I moved configration yamls and kustomization.

etcd debugging flowchart, copy the flowchat from “Stories from the Playbook” for easy reference and put here to make it searchable in my site. flowchart TD oversized{MVCC DB oversized}--|Yes|logIntoContainer(log into container) logIntoContainer -- checkSize(check size of db) checkSize -- compatOrDefrag(compat or defrag) compatOrDefrag -- resizeDisk(resize machine disk) resizeDisk -- triggerRepair(Trigger repair) triggerRepair -- END oversized --|No|crashLoop{crash looping} crashLoop --|Yes| moreTime2Init(allow etc more time to init) moreTime2Init -- upgradeVersion(upgrade version) upgradeVersion -- resizeDisk crashLoop --|No| leaderElectionIssue{Leader Election Issue?

Yesterday I spent one or two hours to resolve a pending order issue. I encountered the issue before when I configured cert-manager with ACME. However after a short while detour to flutter(ios,android) development, I couldn’t quickly locate the root causes. This made me think that it is better off to note down the lessons I leant here. DNS01 vs HTTP01 HTTP01 is quite easy to setup for one domain name.

I can make it works when using device_preview and getx GetMaterialApp. GetMaterialApp can perfectly works with device_preview on many things such as size, orientation etc. GetMaterialApp.router is not the case. I found that locale changes in device_preview can’t propagate to GetMaterialApp.router. I raised a issue, however getx has a bad reputation on fixing issues. I can’t wait so long and I sought other options. Things I tried was to use GetMaterialApp, however always to get several errors.

Terraform site gives several use cases and comparison with other IaC langauges. I think there are few things of terraform that stand out when comparing it with other tools such as cloudformation, ARM tempaltes IaC languages. multi-cloud deployment When you only use one cloud-provider’s solution to do IaC and things get in your way, you will have hard time to migrate to other providers as you already invested heavily into one provider.

void main() => runApp( DevicePreview( enabled: !kReleaseMode, builder: (context) => MyApp(), // Wrap your app ), ); class MyApp extends StatelessWidget { final botToastBuilder = BotToastInit(); @override Widget build(BuildContext context) { return GetMaterialApp.router( useInheritedMediaQuery: true, locale: DevicePreview.locale(context), builder: (context, child) { child = DevicePreview.appBuilder(context, child); child = botToastBuilder(context, child); return child; }, theme: ThemeData.light(), darkTheme: ThemeData.dark(), home: const HomePage(), ); } } Reference: https://pub.dev/packages/bot_toast https://stackoverflow.

I thought it was every easy to setup and use it using STS and gradle. The actual story is not so. I changed my settings followed Developing with Spring Boot and some suggestions here in stackoverflow, however liveload and autorestart don’t work. After several rounds of trial, I noticed spring boot dashboard. I start my application from there, liveload and autorestart work now. developmentOnly("org.

In the past two days, I always got the errors "packages cannot be resolved" in eclipse with gradle projects. Initial I encountered the issue, I desperately wanted to resolved the issue. I tried several ways to do that by using solutions from web, however I didn’t get my issue resolved. Due to time constraint, I went back to the old mvn way to do the project. Today I have sometime to revisit the issue now.

I wondered why I was asked twice about network troubleshooting between two machines and the destination machine is not reachable directly. It seems there are somethings I didn’t understand fully or I missed something. Today I googled "tracert is not enough" and found Traceroute Limitations Explained. It explains my doubts now. Traceroute Limitations Explained gives some limitations of traceroute and introduce their production to avoid those limitations. Are there any other ways?

The real differences is quoted here. You can find the content here A kustomization that is marked as a Component has basically the same capabilities as a normal kustomization. The main distinction is that they are evaluated after the resources of the parent kustomization (overlay or component) have been accumulated, and on top of them. This means that: A component with transformers can transform the resources that an overlay has previously specified in the resources field.

Today I was asked which framework was used in my design of microservice systems. I answered those concerns of microservice are handled natively by kubernetes when my systems are deployed into kubernetes clusters. I will not use those frameworks in my design of microservices. The question sparked several questions in my mind. Here are those questions. Why do we still need microservice frameworks when kubernetes is the de-facto platform now?

In normal kubernetes metrics, you will find cAdvisor, Metrics Server, Kubernetes API Server, Node Exporter, and Kube-State-Metrics as source of metrics. In last 2 days, I noticed prometheus-adapter. Today KEDA popped up in my mind. I checked its website and found it can be source of metric as well. cAdvisor, Metrics Server, Kubernetes API Server, Node Exporter, and Kube-State-Metrics can be considered as in-cluster source of metric. However KEDA is about out-cluster source of metrics.

Recently I added Crashlytics into my project on Oct 25, 2022. I encountered this issue and spent a while to fix it. Today I had to figure out how to do it again, and spent a while. I hated to do that again, so here is this article. The key message is the first one below. It is quite misleading and lead me to wrong direction to change DEBUG_INFORMATION_FORMAT.

Last three days, I found an issue can only be reproduced in realease mode.The beta testing cycle via testflight takes too long and it definitely is not the option. I debugged my application in iOS before. One thing popped up in my mind whether I can put release mode of my application into iOS using the debug tool of vs code. I changed my launch.json as below in my project and debug my application.

Today when I leant SRE course from cloudskillsboost, the first course is google cloud foundation. SRE is great. cloudskillsboost should be best place to learn SRE as it is originated there.

Today I re-read Technology Radar Volume 26, SDUI got my attention. In the past, I was very interested in ERP systems, and studied several open source ERP systems such as openerp(now odoo), ofbiz and adempiere. SDUI is quite similar to technologies used in those open sourced ERP systems. technology data source ui complex SDUI server end logic decides/graphQL odoo

One of my past projects is based neural network in 2020. The solution is quite simple: put the feature vectors into postgresql database create distanced based index I adopted the model and algrithm from face_recognition, and my team trained on sample faces from our internal system to improve the accuracy to reduce the human involvement and save costs. During development, we found that there is a limit of postgres: CUBE_MAX_DIM, my team customize the build of postgreql to work around the limit.

The full article "How to Build a Strong Career in Tech" from Thiago Ghisi is at https://www.linkedin.com/pulse/how-build-strong-career-tech-thiago-ghisi/. I want to put the gist here for my reference. How does a Successful Career in Tech look like? Definition of a Successful Career The candidate resume metadata doesn’t often lie resume metadata: Impact of their ideas on a large number of people directly or indirectly. Success = Impact & Influence!

iPhone Enable developer mode: Setting -→ PPrivacy & Security -→ Developer Mode IDE Android Studio It is quite straight forward in Android Studio. Install flutter plugin and restart Android studio VSCode select the Run and Debug icon in the Activity Bar Select "Dart & Flutter" in "RUN AND DEBUG" dropdown list Edit flutter iphone setting Change launch.

It is quite easy to refactor code using IDEs, however it will be a different story considing to refactor to testable code. Concerns The refactor functionality in IDE is based on rigor theories and algorithm to make sure the refactors don’t change the behaviours of programs and get it better organized. One of the theories, can be found here Target Refactor code to testable codes following best practices.

There are 27 issues with podfile dependency issues of google_mobile_ads in its repository googleads-mobile-flutter at github Here you can find those issues. My team encountered this issue in the past weeks. Today I had the issue again. To save my time in future, I kept my approach here this time. Use the following search keywords in cs.github.com path:pubspec.yaml google_mobile_ads firebase_analytics locate a good candidate with appropriate version of combinations of the following components.

In the last week, I intermittently worked on how to compile a flutter project conditionally like the way c/c++ in busy daily work schedules. I knew that flavor is the way to go, however there are still serveral issues opened to explore to make it better or to understand the rationale behind the scene. Dependency Injections using get_it and injectable: Tree Shaking Capability Pack only neccessary components into the mobile app:

My first puzzle game, implemented in flutters. I intend to integrate github action, fastlane, google adMob, google analytics and firebase. sudoku

Serveral months ago, I practiced serveral binary leetcode questions, and I was confused about the following several things: I didn’t pay much attentions to it then, I thought I will get the hang of it after serveral practices. However after serveral practice, I still confused in binary search problems. Latest week an article mentions binary search at https://leetcode.wang/. Little searching, I found more information about this such as lower bound, upper bound, equal-range, open/close range, middle point etc.

Shift-left is a things nowadays. Recently I read an articles about that beyond testing and security. Today I gave it a try using datree and found that there are still a lot of places to improve in one of my clusters. It is not just a small number. There maybe are many new concepts, configurations and best practicies behind the numbers. kubectl datree test > test.log rg -n ❌ test.

Kuberneters dashboard doesn’t give enough information about node and cluster information during rececent loading test. I sought to other options. Prometheus and grafana are the de-facto standards. It’s a no-brainer choice. The most important things is how to make them working together. Setup Prometheus and Grafana kubectl create namespace monitoring helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo update helm install prometheus prometheus-community/prometheus -n monitoring helm repo add grafana https://grafana.github.io/helm-charts helm install grafana grafana/grafana -n monitoring kubectl get secret --namespace monitoring grafana -o jsonpath="{.

Today I had a case to expose serveral kubernetes dashboard with cert-manager. Initiallly I thought it should be quite easy to setup, but the reality was quite different. My intial yaml is as following. apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-global-dash namespace: kubernetes-dashboard labels: name: ingress-global-dash use-http01-solver: "true" annotations: cert-manager.io/cluster-issuer: "test-issuer" spec: ingressClassName: nginx rules: - host: "dashboard.example.com" http: paths: - pathType: Prefix path: "/" backend: service: name: kubernetes-dashboard port: number: 80 #later changed to 443 according to port of kubernetes-dashboard svc tls: # < placing a host in the TLS config will determine what ends up in the cert's subjectAltNames - hosts: - dashboard.

According to many articles one the web, the way to set timezone is as following: cp /usr/share/zoneinfo/Asia/Singapore /etc/localtime echo 'Asia/Singapore' > /etc/timezone You can indeed get the correct time if you run it in dotnet core 5 docker linux containers. However you couldn’t get the correct local time in dotnet 6.0. I found the issue About time incorrect of docker image about aspnet-6 #62545. After I tried serveral combinations mentioned in the issue, finally I got a working solution.

Prologue I never thought I will use nginx extensively in systems I involved after I found that nginx ingress has some limitations comparing to istio in 2021. Nginx is more than reverse proxy or webservers which I thought initially after several months of extensive usage in the past. Here I list serveral advanced use case I countered or discussed with other experts. CDN MaxCDN use https://docs.nginx.com/nginx/admin-guide/content-cache/content-caching/

I only noticed the steps after one ergent cancelling of creating. I didn’t find corresponding nodes and wondered what happened. I deleted my cluster and recreated one. I knew I can create nodegroup after creating cluster and not sure if I can do those sub tasks this way as well. Leave it here for later investigation. 2 sequential tasks: { create cluster control plane "uat-local-intl", 2 sequential sub-tasks: { 4 sequential sub-tasks: { wait for control plane to become ready, associate IAM OIDC provider, 4 parallel sub-tasks: { 2 sequential sub-tasks: { create IAM role for serviceaccount "backend-apps/s3-reader", create serviceaccount "backend-apps/s3-reader", }, 2 sequential sub-tasks: { create IAM role for serviceaccount "backend-apps/cache-access", create serviceaccount "backend-apps/cache-access", }, 2 sequential sub-tasks: { create IAM role for serviceaccount "kube-system/cluster-autoscaler", create serviceaccount "kube-system/cluster-autoscaler", }, 2 sequential sub-tasks: { create IAM role for serviceaccount "kube-system/aws-node", create serviceaccount "kube-system/aws-node", }, }, restart daemonset "kube-system/aws-node", }, create nodegroup "ng-1", } }

In the past I knew jsonnet, however I didn’t commit too much time to learn it until recently I encountered a data driven scenario, generating istio yaml configurations based on an array of data. The scenario is business driven, and I can foresee that there will be more data coming. What a waste of time to repeat the copy/paste procedures with little tweaks. I hated to do that again and again.

I started one of my certificates using cert-manager and letsencrypt without root domain in one product system. Later I tried to add root domain to the certificate as our customer requested withou success at first day. I figured out how to do that next morning. Here are my the success one and several failed attempts. Here some background information of this system: gitops + argocd Successful solution Considering that the app argocd is in manual synchronization mode, I can do that changes in place in kubernetes instead of the git repo.

In the past 1 week, I encountered an issue that I couldn’t add a column to a table in one of our production database. I thought restart would resolve the issue. After a restart, however the issue still persist. I checked logs in error/mysql-error-running.log in rds and noticed there are two transactions in prepared state after recovery. In the first gooogle search result page, How to Deal with XA Transactions Recovery caught my attention.

ChatOps has its place, even more prominent with AI and rich integration with other systems. I learnt ChatOps in the past when I first heard hubot. Later GitOps and Kubernetes grabbed my attention. I didn’t see how ChatOps fit in GitOps until I saw this 湾区日报是如何运作的？ today after I implemented an integration with telegram bot APIs. What Are the Benefits of ChatOps? enumerates the benefits. In my eyes I saw other benefits or more benefits in productivity when considing the full potentials of ChatOps.

TLDR: my CICD choice: argo-cd and argo-workflow Since I chose argocd and github action as our CICD pipeline two years ago, I rarely thought about other choices and the reason I made the choices. Recently I leant from several sources that jenkins is still widely used, it arouse my interest on this subject again. I reconsidered my choice and layed out the reasons here for my future references. I didn’t have a good impression with java in my past java projects.

Since my first time to configure istio with proxy protocol supports in aws, istio and envoyproxy changed a lot. In the past several days, I was exposed to a different scenario that one service to expose into multiple domains. Things are getting not straighforward, I struggled to make it success yesterday. Today another layer of complexity was added into my cluster: an extra ingressgateway for some services. For multiple ingress gateways, I followed this article, however my setup is more complex than that.

Turn off "Block public access (bucket settings)" Access control list (ACL): give Everyone (public access) object lists and bucket ACL read. Policy Policy to attach to a role, so the role can be used to upload resources and put the ACL on the uploaded objects. { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::demo" }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::demo/*" }, { "Sid": "VisualEditor2", "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::demo-public" }, { "Sid": "VisualEditor3", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject", "s3:PutObjectAcl" ], "Resource": "arn:aws:s3:::demo-public/*" } ] } Uploading command line bash scripts aws s3 sync --acl public-read local-dir/ s3://demo-public/test/ C# source code using (var client = new AmazonS3Client(RegionEndpoint.

When I tried to create an user for application instead of giving out the master user and its password. I got "Error Code: 1045. Access denied for user 'mysqladmin'@'%' (using password: YES)" when I run "GRANT ALL PRIVILEGES ON demodb.* TO 'appadmin'@'%';". I tried the method in How do I create another master user for my Amazon RDS DB instance that is running MySQL?, it is for creating another master user with more permissions than an app db user.

I got following similar errors when setup my istio clusters. Mark bundle as not supporting multiuse 301 istio 301 or 404 error:02FFF036:system library:func(4095):Connection reset by peer * Trying 20.190.14.28:443... * TCP_NODELAY set * Connected to kiali.example.com (20.190.14.28) port 443 (#0) * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /home/ng/anaconda3/ssl/cacert.pem CApath: none * TLSv1.3 (OUT), TLS handshake, Client hello (1): * OpenSSL SSL_connect: Connection reset by peer in connection to kiali.

Today I explained the current infrastructure fleet configurations to our devops team, I have the thought to make a reflection on the things or lessons I learnt in the past and think about what should I do in the next. The technologies and tools I used in my projects is as following. There are still a lot of things to learn considering the list in my list and the huge ecosystem of kubernetes.

Here is an issue with aws-ebs-csi-driver: The size of file system doesn’t change when pvc is expanded. I got the same issue when I tried to do the Curl elk in pods to delete indices this afternoon. I got the message "resize2fs 1.44.5 (15-Dec-2018) open: No such file or directory while opening /dev/nvme1n1" as well when I tried to resize the file system /dev/nvme1n1 in my pod. As the issue is about csidriver, it is not in the the result of running command "kubectl get csidriver" on my cluster.

Today my staging kibana didn’t show logs. I made the decision to work out a solution to solve issue in hard way this time. I don’t want me in the same situation without solutoins. When things go wrong, you can’t login kibana to do management or maintainance works. The left option is managing the data from the command line. In the past I figured out to use curl cli in pod to get some information of elk.

As in my previous article, I gave the following information of my pod. I still have some time before bed, I couldn’t help to seek the reason of that. /usr/share/nginx/html # df -h Filesystem Size Used Available Use% Mounted on overlay 80.0G 34.5G 45.5G 43% / tmpfs 64.0M 0 64.0M 0% /dev tmpfs 3.7G 0 3.7G 0% /sys/fs/cgroup /dev/nvme0n1p1 80.0G 34.5G 45.5G 43% /dev/termination-log /dev/nvme0n1p1 80.0G 34.5G 45.5G 43% /etc/resolv.

I did resize volume of kubernetes in the past, however I encountered an interesting issue when I did the resizing in different way. According to doc, I should only change the requested size in pvc. Today I changed the size of pv first, then pvc. Here was the interesting thing: all things of pv and pvc are fine, but the size of the file system in pod was not changed.

Today I needed to scale one of my kubernetes clusterss. Those keys are not in my new laptop since I used Mac Air M1. I didn’t see the increase of nodes several minutes after I run the eksctl scale command. I logged into aws console and found there were several "Failed" messages in activity history of the autoscaling group. All the failed messages shown "Launching a new EC2 instance. Status Reason: The aaaa-nodegroup-ng-1-67:8e:b8:8e:33:83:93:68 key pair does not exist.

I wrote this on September 7, 2021 and published it on linkedin. However I found that it can be hard to search if I put there. so I put here anoter copy TLDR: if your external SMTP is using port 80 instead of standard ports in an istio mesh, create a Service Entry for the external SMTP. These two days, a strange timeout issue happened in one of our kubernetes clusters when trying to send emails via SMTP even though the same configuration works perfectly on our development machines.

Summary: use digest as the way to refer to docker images in kubernetes resources, put commit id in image labels. An idempotency and immutable infrastructure has a slew of benefits. I am a firm believer of it and I did my best to keep several projects in that way. In the past few weeks, I helped one friend to resolve system structure, performance issue and development experience of one of his projects.

Yesterday I added gitconfig volume and mounted to /root. However today the logs of the workflow shows netrc is mounted to /root as well. They are conflicted. Kubernetes supports projected volume which supports following types. my netrc and gitconfig are in the list. secret downwardAPI configMap serviceAccountToken I made the small tweak to hello world sample workflow. Here is the my demo workflow with projected volumes.

Since I read the textual UML in the 2015 techradar, I am a big fan of it. I used plantuml in my technical writing in Asia Fusion Technology. Later I found asciidoc, and I wanted to combine the two together. I regularly read techradar from thoughtworks and found mermaidjs in NOV 2018 techradar. However it was little hard to integrate that into my site at that time. Today I knew Hugo supports Mermaid Diagrams easily.

Prologue Today I migrated several golang projects to golang 1.18. I had several hiccups today. Let’s called docker buildx multi-context, multi-platform, multi-stage and secrets in action. Multi-platform I started to look into one iOS and android app of our projects one week ago, and started to use Mac Air M1. Mac Air M1 is arm64 based, however our production environment is amd64 based. I had to do some changes to make the images built on my or my colleagues' machine can be runnable in production environment.

I had some projects on github, gitlab and gitee. Their CI pipelines are quite different. To save the troubles and times, it is a natural choice to seek other options to unify on one solution. At first, I looked into jenkins x. However it is quite heavy and with opinions. Considering its Maturity Level Matrix, I am not sure I can make it happen in my limited after hours times.

I needed to clone one of my private repositories in one workflow of argo-workflows. I did’t figure a way to mount git artifact into a shared volume at that time. I sought to other options and find https://github.com/argoproj/argo-workflows/issues/1428 and https://www.jeffgeerling.com/blog/2019/mounting-kubernetes-secret-single-file-inside-pod. The comment https://github.com/argoproj/argo-workflows/issues/1428#issuecomment-775911838 gives an example to add user name and password into https url. Upon I checked the logs, I found that user and password are shown in the logs of a workflow.

Today I found my argo-events sensor executed triggers which I was not intended to. In the logs, I found that ref fields of those non-intended triggers all started with "refs/heads/dev". I checked again with document and found this surprising truth: https://argoproj.github.io/argo-events/sensors/filters/data/#:~:text=If%20data%20type%20is%20string%2C%20you%20can%20pass%20either%20an%20exact%20value%20or%20a%20regex.%20In%20any%20case%20that%20value%20will%20be%20evaluated%20as%20a%20regex. If data type is string, you can pass either an exact value or a regex. In any case that value will be evaluated as a regex. Considing my following data filter, no wonder it triggers more than I expected.

Webhook of argo-events supports authentication, however the field is fixed in header. If your callers don’t do that way, you are screwed. Recently days, I was trying to integrate gitee with argo-workflow and argo-events. I hit the wall here. TOKEN="Bearer af3qqs321f2ddwf1e2e67dfda3fs" curl -X POST -H "Authorization: $TOKEN" -d "{your data}" http://xxxxx:12000/example I kept reading the documents for serveral days and the knowledge sinked in, I got the idea that filters maybe can be used for that as a workaround solution.

I heard from my colleagues that MacAir M1 is picky about mice and you can find many threads in reddit about this if you use the keywords "reddit mac air m1 mouse issues bluetooth" to search. Today is my lucky day, I encountered the issue too. This morning, I brought my Mac Air to office and used the mouse of company. The bluethooth kept spinning and the mouse can’t connect.

Low resource usage Our system had low resource usage, however my several pods got stucked in ContainerCreating state. Kubernetes events show that "add cmd: failed to assign an IP address to container". "kubectl top nodes" still shows the usage of resources still low. jackl@LAPTOP-IIHUF2DR:~$ k --context jack.l@test-dev.ap-southeast-1.eksctl.io top nodes NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% ip-192-168-102-109.ap-southeast-1.compute.internal 143m 7% 3217Mi 45% ip-192-168-110-171.ap-southeast-1.compute.internal 117m 6% 2892Mi 41% Migrate to calico Recreate cluster without nodegroups

The journey I worked on a dapr microservice project which need to read a list from a file and do some useful things. Argo-workflow supports loop over a list of items, however one of my steps always got "invalid character 'G' looking for beginning of value" before going to next step in the workflow template. The step is as following: - name: get-deps inputs: parameters: - name: changes script: image: ubuntu:latest command: [bash] #args: ["echo result was: {{inputs.

Two days ago, I have a fun journey to find more about rich link preview features in social media, web apps and IM apps. The journey began with that the shared link of one of my company sites in skype was shown without pictures, just a plain link. Searching the feature of skype "link preview", I found https://socialsharepreview.com/. I knew what missed in the homepage as in the following list.

Usually I don’t want to blog about something, espicially I don’t contribute some original things to internet. Of couse, there are other reasons as well. There were many reasons which stopped me in the past to writing. One was that my vocabulary was limited. I started my blogging actitivies back to 2004. At that time, my vocabulary was limited, I think the size of my vocabulary was in the range from 6000-7000 and most words were memorized in isolated way.

I wrote my blogs using Jekyll at this domain almost two years occasionally from 2018 until 2020. I didn’t think to move to Hugo before it was used in one of my company project. After two or three weeks since the project kicked off, I had a thought why not move to Hugo considering several benefits. For the uninitiated, Hugo is one of the most popular open-source static site generators.

Today I encountered a strange issue with one kubernete cluster. Newly created pods are not created on a newly joined node. There is no errors in logs of pods and the node, events of kubernete. All seems normal. I created a simple nginx pod with nodeName as the problematic node. The pod can be scheduled and created in the node. However the pod was kept in CrashloopBackOff state. There is definitely something wrong with server.

Dapr development is quite difficult to setup correctly. You can get the idea from (this issue) as the issue is still open now since it was created. I followed the instruction to install dapr vs code extension in dapr. My environment is WSL based. I tried several times to try to debug the dapr applications and failed. I noticed in the following screenshot that there was a warning icon before kubernetes.

Keep WSA running WSA seems sometimes will exit after a while idle time. I need to run the following command to keep it running. adb connect YOUR_IP_ADDRESS Run App According to How to use Windows Subsystem for Android as a development tool, there are several ways to run flutter apps in WSA. Android studio When I try to run a project of my company in Android studio, I kept get the error "An unexpected packet was received before the handshake".

TLDR: Bazel is definitely worth to use on your projects even though it has minimum supports of some languages such as C#. Prologue Before I was interested in bazel, I always thought why bother to learn a new building system even I knew that kubernetes once was built with bazel. 4 months ago, I was involved in a micro service structured project. I was dissatisfied with dev workflow using skaffold and I looked other options in the market.

Install WAS Upgrade to windows 11 latest version (GA, Preview, Evaluation etc). Upgrade microsoft appstore to latest version Setting→Time & Language → Langage & Region → Region: USA Open AppStore, Search Amazon Appstore and install Amazon Appstore. it will install Windows Subsystem for Android as well. Install applications as user using Amazon AppStore Register as Amazon US user

Last week I setup our staging kubernetes environment, however the once worked version of runnerset yaml configuration didn’t work. I tried different combination of controller and summerwind/actions-runner-dind image, fully customized template portion with RUNNER_xxx arguments. I still got "Http response code: NotFound from 'POST https://api.github.com/actions/runner-registration'" for every combination of those combinations in the logs of runner pods. I thought RunnerDeployment is stateless and less complex than RunnerSet. I gave it a try.

TLDR: if your external SMTP is using port 80 instead of standard ports in an istio mesh, create a Service Entry for the external SMTP. These two days, a strange timeout issue happened in one of our kubernetes clusters when trying to send emails via SMTP even though the same configuration works perfectly on our development machines. At first, I thought it was due to SecureSocketOptions.startTLS. I changed it to SecureSocketOptions.

Two days ago, our web applications need to get clients' real IP address. Our sites is hosted in kubernetes, isito is used for ingress controll. behind cloud load balancers, it is not easy to make it work. There are two ways to do that in alibaba classic load balancer: http/https listeners, and TCP listeners with proxy protocol HTTPS listeners is crossed out due to certificate issue. HTTPS listeners can use integrated certificates or self uploaded certificates.

Download opencv release package https://github.com/opencv/opencv/releases/download/4.5.2/opencv-4.5.2-vc14_vc15.exe run the downloaded executable, and say install it at c:\software\opencv run opencv\build\setup_vars_opencv4.cmd After above steps, your environment is ready. However there is a caveat, the environment is ephemeral.

Private Internet Access Desktop Client window dev environemnt https://www.qt.io/download-qt-installer How to find the Qt5 CMake module on Windows [1] 1. https://github.com/pia-foss/desktop

Tryton WSGI dev: werkzeug prod: µwsgi[1] Deployment: nginx for static assets and tls termination, µwsgi for service HA & Load Balancing race-condition[2] cron: distributed ? Training https://github.com/coopengo/tryton-training 1. https://discuss.tryton.org/t/wsgi-server-choice/2766/3 2. https://discuss.tryton.org/t/multiple-tryton-server-backend-with-nginx-frontend/3843

Secrets Management is the strong suit of Hashicorp vault. Open source edition is little limited comparing to cloud and enterprise edition, and cloud and enterprise editions sometimes are not allowed due to some reasons. bank-vaults seems a promising solution based on open source edition of Hashicorp Vault. Advanced usage of vault will involve HSM and SoftHSM is good start point without spending several hundards bucks. Besides, HSM is widely used in Financial sector, blockchain and cryptocurrency.

2 weeks I started to learn alicloud to prepare the migration from aws to alicloud. I started the migration this week. Our applications are deployed on kubernetes, I focused on ACK first. The setup journey was bumpy, and most of the issues can be resolved by googling and trial loops. The most difficult one is that dns resolution issue. I tried the steps in "Debugging DNS Resolution" [1]. All steps were good except the nslookup step.

Add command line flags in ingress controller: --tcp-services-configmap --udp-services-configmap Create configmap apiVersion: v1 kind: ConfigMap metadata: name: tcp-services namespace: ingress-nginx data: 5432: "default/postgres:5432" Patch the ingress-nginx-controller to allow port 5432 spec: template: spec: containers: - name: controller ports: - containerPort: 6379 hostPort: 6379 Add inbound rule to security group of node groups Reference: https://minikube.

Several months ago, I resolved the issue "404 Error on refresh with SPA" by using nginx try_files directive, this time it was quite different as the apps were deployed into kubernetes. I tried nginx.ingress.kubernetes.io/use-regex:true and nginx.ingress.kubernetes.io/rewrite-target: /$1. To avoid to block others to others to use staging environment, I setup an skaffold local development environment to troubeshooting the issue. I tried following annotation as well. nginx.ingress.kubernetes.io/configuration-snippet: | try_files $uri $uri/ /index.

var spec = 'https:\/\/raw.githubusercontent.com\/vega\/vega\/master\/docs\/examples\/bar-chart.vg.json'; vegaEmbed('#viz', spec).then(function(result) { }).catch(console.error);

var myChart = echarts.init(document.getElementById('echarts400')); var option = JSON.parse("\n {\n \"textStyle\":{\n \"color\":\"#fff\"\n },\n \"title\":{\n \"text\":\"2018年11月国内浏览器数据统计\",\n \"subtext\":\"浏览器数据分析\",\n \"x\":\"center\",\n \"textStyle\":{\n \"color\":\"#FFffFf\"\n }\n },\n \"tooltip\":{\n \"trigger\":\"item\",\n \"formatter\":\"{a} \u003cbr/\u003e{b} : {c} ({d}%)\"\n },\n \"legend\":{\n \"type\":\"scroll\",\n \"orient\":\"vertical\",\n \"right\":10,\n \"top\":120,\n \"bottom\":20,\n \"data\":[\n \"Chrome\",\n \"IE 9.0\",\n \"IE 11.0\",\n \"QQ\",\n \"IE 8.0\",\n \"2345\",\n \"搜狗高速\",\n \"Firefox\",\n \"Safari\",\n \"其他\"\n ],\n \"textStyle\":{\n \"color\":\"#fff\"\n }\n },\n \"series\":[\n {\n \"name\":\"浏览器用户比例\",\n \"type\":\"pie\",\n \"radius\":\"55%\",\n \"center\":[\n \"50%\",\n \"60%\"\n ],\n \"data\":[\n {\n \"name\":\"Chrome\",\n \"value\":46.88\n },\n {\n \"

During practicing the lifecycle handlers of pod, I found it is quite difficult to get the logs logged from preStop handlers. Initially I thought terminationMessagePath maybe is the answer, but no such luck. The preStop handler will be run before the pod is deleted. After deleted, there is no way to get the logs unless those logs are kept in other place such as central log servers etc. Those settings are way complex for a simple practice and time-consuming.

python-guestbook at https://github.com/GoogleCloudPlatform/cloud-code-samples is used to practice debug python containers in kubernetes. Google cloud python code vscode plugin 1.7.0 failed to run python on kubernetes or there is no obvious way to do that as depicted in the below pic. Steps to make it: run skaffold debug Import ptvsd in the python file to debug, add breakout() before the line to debug

I can’t debug the dockerdev app following the steps in https://blog.jetbrains.com/go/2020/05/11/using-kubernetes-from-goland/, even after I cloned its source code https://github.com/jackliusr/dockerdev/tree/kubernetes-debug. I read several articles before I tried the aboved one. The successful debug after several trial and errors. I added skaffold.yaml to the project and installed Cloud Code. skaffold.yaml can be found at https://github.com/jackliusr/dockerdev/tree/kubernetes-debug. Other jetbrain configuration can be seen in the below pictures. cloud code is a very good tool.

I loved to read novels. Recently I found that several sites show some garbed characters in square boxes. I went to the original site qidian and those chacters are shown correctly in the orignal site. After little investigation, I found the following information; In above pics, 了’s HTML entity code is , it is not as 天. How does chrome show the characters correctly?

# https://www.kubeflow.org/docs/pipelines/installation/localcluster-deployment/#deploying-kubeflow-pipelines # env/platform-agnostic-pns hasn't been publically released, so we install from master temporarily export PIPELINE_VERSION=1.0.1 kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION" kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-pns?ref=$PIPELINE_VERSION" # expose gui kubectl port-forward -n kubeflow --address 0.0.0.0 svc/ml-pipeline-ui 8080:80

KIND supports configuration items and you can find them at https://kind.sigs.k8s.io/docs/user/configuration. Following is the outline of the doc. Cluster-Wide Options ︎ Networking: ︎ IP Family ︎ API Server: port and listen address Pod Subnet ︎ Service Subnet ︎ Disable Default CNI: can be used to try other CNI for CKA and CKAD networks kube-proxy mode : iptables, ipvs Nodes: role control-plane, worker Per-Node Options ︎ Extra Mounts ︎ Extra Port Mappings ︎ Kubeadm Config Patches: Kubeadm Config Patches, kubeadm InitConfiguration and JoinConfiguration My asciicast is as following:

Today one of my deployments always got CrashLoopBackOff. Searching internet don’t gave me the root cause of those errors even I went through all the steps. Steps from internets are all about describe, logs, liveProbe etc. I noticed the reason and exit code in last state when I described one of the pod. It should be the focus point to find root cause of CrashBoopBackOff. Containers: nginx: Container ID: containerd://9570e7e67d83692fdbe0e0871919a81222137fcaee2eaecb1eff6b772ec805b1 Image: nginx:1.

I am a big fan of Tex and asciidoc. Asciidoctor is ruby based and can been easily integrated into jekyll. Hugo currently still has some rough edges in asciidoc integration. Considering the time and effort to switch to Hugo, it is not worth to do that.

run redux-saga examples TL;DR: yarn yarn build I run into module not found issues when I try to run those examples under the repository of redus-saga. Even following comments in github issues and stackoverflow, those solutions don’t work out. Scripts in the package.json of the root folder don’t work for examples. After run yarn && yarn build, can change directories to those examples and run them.

The difference of two squared numbers $ a^2 - b^2 = (a + b )(a - b) $

I was not familiar with SIP states during the integration of VoIP products such as twilio, plivo and sinch. I am still confused about the state changes after reading documents. Here try to visualize the state change and help me to remember it. The state chart is based on my understanding. It is not necessarily correct. It is just for my reference. QueuedInitiatedRingingAnsweredInProgressCompletedBusyNoAnswerCanceledFailed

Measurement Physical properties SI Units Other Common Units Common measuring apparatus Name Symbol time second s hour (h) minutes(min) day digital stopwatch analogue stopwatch mass kilogram kg gram(g) tonne(t)

PlantUML Diagram Class diagram—​Relations between Class: Type Sysmbol Extension <!-- Composition *-- Aggregation o-- Asciidoc UML sample Class01Class02Class03Class04Class05Class06Class07Class08Class09Class10 PlantUML sample Class01Class02Class03Class04Class05Class06Class07Class08Class09Class10

Asynchronization is very hard. It brings benefits with several catches for backend programming. Here is serveral catches and respective solutions. However all those will increase the cost of developments. Hard to debug and trace As the process breaks down to serveral asynchronious subprocesses, it is very difficult to relate all information of the process when something goes wrong. An extra overhead correlation Id is needed to adopt to ameliorate this issue.

Ditaa 0 start1 q scans from start to end.....endrest of theallocated memory.....n