Traditionally, batch processing of files has been the default approach in many on-premise solutions. Files are dropped into a directory, collected on a schedule, and processed in groups. This model works fine for systems where latency is not critical, but once you start asking “How do we make this real-time?” the story becomes more interesting. At first, my answer was polling. After all, if we want near real-time, we can just keep checking the directory at short intervals.

Usually one will use simple find or more advanced regular expression based search to locate codes. How about use other options usig ast to get more accurate results and have the potential to use for codemod? In the past 2 days, I finally figured how to use ast-grep to find code. Here I will use it to find all extension methods of an class. The rule is configured below.

Introduction In the world of modern software development, Shift-Left Security and DevSecOps have become de facto practices. The idea is simple: security should not be an afterthought but rather an integral part of the development lifecycle, starting as early as possible. While these practices strengthen security and compliance, they often introduce additional steps that can slow down the development pipeline. This raises an important question: Is there a way to speed things up without sacrificing security?

Immutable infrastructure is a concept that has revolutionized the way we think about deploying and maintaining systems. It ensures that once an instance of infrastructure is created, it is never modified. Instead, any changes require the creation of a new instance. This approach enhances consistency, reduces configuration drift, and simplifies rollback processes. While immutable infrastructure is often associated with Infrastructure as Code (IaC) and containers, it’s entirely possible—and sometimes necessary—to implement it with Virtual Machines (VMs) instead of containers.

REM download https://github.com/containers/podman/releases/download/v5.2.5/podman-remote-release-windows_amd64.zip REM unzip it to C:\DIM\podman-remote-release-windows_amd64 SET PODMAN_HOME=C:\DIM\podman-remote-release-windows_amd64\podman-5.2.5 set PATH=%PATH%;%PODMAN_HOME%\usr\bin podman machine init podman machine start set DOTNET_ASPIRE_CONTAINER_RUNTIME=podman Get podman desktop from https://podman.io/

A newly created postgresql container will do a following steps. They are quite time consuming in integration testing and add some unpredictable factor. If a integration testing will clean up tables and data, volumes can be used to speed up postgresql container. pg_ctl -D /var/lib/postgresql/data -l logfile start initdb stop postgresql and start again There might be one issue, do I need to maintain the same credential?

Today I read an article Learning to learn and it mentions the concept "expert beginner" I learnt in past. I am wondering the differences among other concepts (namely generalizing Specialists, icicle-shaped vs t-shaped) in those domain, where I fall into the category, do I do the correct career choices etc. The differences among them can be quickly generated from chatgpt, I quoted the answers here for quick references. Aspect Expert Beginner T-Shaped Professional Generalizing Specialist Icicle-Shaped Specialist Skill Depth

Since wrote Things learnt from recent preparation of 1T data for testing, I had figured out how to do that on the following weekends. Today I couldn’t get the steps to do so immediately. It is better to note down for my next time usage. The detailed steps is as following Find the location in the file Get the block contains contents to modify Extract the block to a file and modify it

Several things learnt from recent preparation of data for testing. Question the way when it takes too long to process data sed: remove last n lines In the past I always thought sed is the fastest way to manipulate data. Is it true for all the cases? I generated 1T testing data in ndjson format, however the last lines of those files are corrupted json data. I had to remove the last line.

One thing constantly bothered me since I used mermaid for flowchart, how to correctly group elements in a group. Only today I have a thought about what will happen if I put node in subgraph without links between them. Finally I got it works to correctly group elements in a group. Another things, mermaid diagram is natively supported in asciidoctor now. This one is my first example to use it.