ELK

May 13, 2020

I was working on adding logs into elastic search from one of our micro API projects. I learned a little bit, that I'll probably forget when I'll deal with this next time. So, I'll write it here.

1. ELK - Elastic Stack. How it works:

• Serilog is client end library. It it catching log entry and can send log directly to elastic search, but (afaik) most commonly send it into logstash. Serilog "sink" represents, where data can go (console, file, tcp, elasticsearch, etc).
• Logstash is some pipeline server side tool that ingest logs from multiple sources, transform them and send into elastic search.
• ElasticSearch is search and analytics engine.
• Kibana is UI for elastic search. Actually, what I wrote here is described here.

2. It's very useful to run ELK Stack local. Of course, installing everything will take too much time (I still remember painy days spent installing lamp). Of course, these days everything is in container and there is one good repository, that will help running everything in minutes. All you need to do is:

git clone https://github.com/deviantony/docker-elk.git
cd .\docker-elk\
docker-compose up

And read ReadMe.md for further details.

3. In my case, we have configured sentry and adding additional logging system, without removing old one. In this case better to remove "Logging" section in appsettings, otherwise it will affect both system. Most tutorials recommend start logging in Program.cs before actual werver will start, but I find it too cumbersome. If server didn't started, we will figure it out by not seing swagger page or by errors from other service, that consume this api.
4. Also, most tutopials did not mention, that serilog can be configured not in code, but in appsettings. Probably, additional libraries needs to be installed, but then startup code itself is not polutted with config lines.
5. When kibana resist to show logs, it means that, probably, indecies were not setted up correctly. And keep in mind, that it takes some time for log to appear, even if everything is running locally.
6. Some usefull links:

• Again, docker compose for elk stack
• one nice tutorial about topic
• another one
• nuget to move serilog configuration into appsetings
• C# logging with Serilog, clear youtube tutorial

This is probably too much info for such easy topic, but I might find it usefull some day later and - hey - I'm blogging again!😊